Optical adjustment methods, head-mounted display devices, and computer-readable storage media

Abstract

This application discloses an optical adjustment method, a head-mounted display device, and a computer-readable storage medium, relating to the field of head-mounted display device technology. The optical adjustment method is applied to a head-mounted display device, which includes a variable-focus lens. The method includes: acquiring data on changes in the wearer's eye state; predicting target optical parameters, including a target radius of curvature, based on the eye state change data using a pre-trained optical parameter prediction model; and adjusting the curvature of the variable-focus lens based on the target radius of curvature. This application enables more precise matching of the focusing state adjustment of the optical system, thereby reducing the VAC effect of the head-mounted display device.

Description

Technical Field

[0001] This application relates to the field of head-mounted display technology, and more particularly to an optical adjustment method, a head-mounted display device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of head-mounted display technology, the VAC (Vergence-Accommodation Conflict) problem has increasingly become a key bottleneck affecting user experience. The VAC effect stems from the mismatch between the display plane of the virtual image and the natural focusing needs of the human eye, causing the convergence of the eyes and the accommodation of the lens to decouple, resulting in visual fatigue, dizziness, and other discomfort, which seriously restricts the immersion and usability of head-mounted displays.

[0003] Traditional head-mounted displays primarily rely on fixed-focal-length designs or manual adjustment by the user for visual accommodation. This mechanism is simple in structure and low in cost, making it widely used in current mainstream devices. However, in practical use, different users have varying focusing needs for clear imaging in different scenarios. Fixed-focal-length or manual adjustment modes offer limited freedom of adjustment, and the accuracy is significantly affected by the user's subjective judgment, making it difficult to consistently ensure consistency between the optical system's output and the user's focusing requirements.

[0004] Therefore, how to more accurately adjust the focusing state of the optical system to reduce the VAC effect of the head-mounted display has become an urgent problem to be solved by those skilled in the art.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide an optical adjustment method, a head-mounted display device, and a computer-readable storage medium, aiming to solve the technical problem of how to more accurately match the focus state adjustment of the optical system in order to reduce the VAC effect of the head-mounted display device.

[0007] To achieve the above objectives, this application proposes an optical adjustment method applied to a head-mounted display device, the head-mounted display device including a variable focus lens, the method comprising:

[0008] Acquire data on changes in the wearer's eye condition;

[0009] Based on the eye state change data, the target optical parameters are predicted by a pre-trained optical parameter prediction model, wherein the target optical parameters include the target radius of curvature.

[0010] Based on the target radius of curvature, the curvature of the variable focus lens is adjusted.

[0011] In one embodiment, the step of predicting the target optical parameters based on the eye state change data using a pre-trained optical parameter prediction model includes:

[0012] The data on changes in eye state are input into a pre-trained optical parameter prediction model to predict the target optical parameters.

[0013] In one embodiment, after the step of inputting the eye state change data into a pre-trained optical parameter prediction model to predict the target optical parameters, the method further includes:

[0014] The target radius of curvature is input into a pre-trained distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model.

[0015] The target distortion parameters are injected into the rendering pipeline of the head-mounted display device.

[0016] In one embodiment, the step of predicting the target optical parameters based on the eye state change data using a pre-trained optical parameter prediction model includes:

[0017] Acquire the wearer's environmental perception depth data;

[0018] The eye state change data and the environmental perception depth data are input into a pre-trained optical parameter prediction model to predict the target optical parameters.

[0019] In one embodiment, after the step of inputting the eye state change data and the environmental perception depth data into a pre-trained optical parameter prediction model to predict the target optical parameters, the method further includes:

[0020] The gaze information of the wearer is obtained, wherein the gaze information includes gaze point coordinates, gaze depth, and convergence angle;

[0021] The target radius of curvature and the gaze information are input into a pre-trained distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model.

[0022] The target distortion parameters are injected into the rendering pipeline of the head-mounted display device.

[0023] In one embodiment, the head-mounted display device further includes a movable grating, and after the step of adjusting the curvature of the variable focal length lens based on the target radius of curvature, the method further includes:

[0024] Acquire the light intensity distribution data of the display screen of the head-mounted display device, and calculate the current light field uniformity information of the head-mounted display device based on the light intensity distribution data;

[0025] Obtain preset target light field uniformity information, and adjust the position of the movable grating based on the target light field uniformity information and the current light field uniformity information.

[0026] In one embodiment, after the step of adjusting the position of the movable grating based on the target light field uniformity information and the current light field uniformity information, the method further includes:

[0027] Based on the position of the movable grating, the distortion parameters of the head-mounted display device are updated to obtain the updated distortion parameters;

[0028] The updated distortion parameters are injected into the rendering pipeline of the head-mounted display device.

[0029] In one embodiment, the step of adjusting the position of the movable grating based on the target light field uniformity information and the current light field uniformity information includes:

[0030] The optical field uniformity error is calculated based on the target optical field uniformity information and the current optical field uniformity information.

[0031] The light field uniformity error is input to a preset PID controller to obtain the grating control signal output by the PID controller, and the position of the movable grating is adjusted by the grating control signal.

[0032] In one embodiment, the target optical parameters include the target lens focal length, and prior to the step of adjusting the curvature of the variable focal length lens based on the target radius of curvature, the method further includes:

[0033] The refractive index of the material of the variable focal length lens is obtained, and the focal length of the target lens and the radius of curvature of the target lens are verified by the thin lens focal length formula based on the material refractive index to obtain the verification result of the thin lens focal length formula.

[0034] If the verification result of the thin lens focal length formula is passed, the following step is performed: adjusting the curvature of the variable focal length lens based on the target radius of curvature.

[0035] In one embodiment, after the step of adjusting the curvature of the variable focal length lens based on the target radius of curvature, the method further includes:

[0036] Obtain the comfort score of the wearer and calculate the rendering quality index of the head-mounted display device;

[0037] The comfort score and the rendering quality index are input into a preset first reward function to obtain a first reward value output by the first reward function.

[0038] Based on the first reward value, the weight parameters of the optical parameter prediction model are updated.

[0039] In one embodiment, the head-mounted display device includes an automatic scoring module, and the step of obtaining the wearer's comfort score includes:

[0040] Obtain the ciliary muscle EMG data of the wearer, and calculate the ciliary muscle tension of the wearer based on the ciliary muscle EMG data;

[0041] Acquire the pupil diameter sampling data of the wearer, and calculate the pupil diameter change rate of the wearer based on the pupil diameter sampling data;

[0042] The ciliary muscle tension and the pupil diameter change rate are input into the automatic scoring module to obtain the first comfort score output by the automatic scoring module.

[0043] Based on the first comfort score, the wearer's comfort score is determined.

[0044] In one embodiment, the head-mounted display device further includes an active scoring module, wherein the step of determining the wearer's comfort score based on the first comfort score includes:

[0045] Obtain the second comfort score submitted by the wearer through the active scoring module;

[0046] The first comfort score and the second comfort score are weighted and fused to obtain the wearer's comfort score.

[0047] In one embodiment, after the step of injecting the target distortion parameters into the rendering pipeline of the head-mounted display device, the method further includes:

[0048] Obtain the comfort score of the wearer and calculate the rendering quality index of the head-mounted display device;

[0049] The comfort score and the rendering quality index are input into a preset second reward function to obtain a second reward value output by the second reward function.

[0050] Based on the second reward value, the weight parameters of the distortion parameter prediction model are updated.

[0051] In addition, to achieve the above objectives, this application also proposes a head-mounted display device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the optical adjustment method as described above.

[0052] In addition, to achieve the above objectives, this application also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the optical adjustment method described above.

[0053] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the optical adjustment method described above.

[0054] This application provides an optical adjustment method, a head-mounted display device, and a computer-readable storage medium, relating to the field of head-mounted display device technology. The optical adjustment method is applied to a head-mounted display device, which includes a variable-focus lens. The method includes: acquiring data on changes in the wearer's eye state; predicting target optical parameters, including a target radius of curvature, based on the eye state change data using a pre-trained optical parameter prediction model; and adjusting the curvature of the variable-focus lens based on the target radius of curvature.

[0055] This application embodiment utilizes data on changes in the wearer's eye state, combined with a pre-trained optical parameter prediction model, to achieve dynamic and personalized adjustment of the optical system of the head-mounted display device. This effectively reduces the accommodative-verbose conflict (VAC effect) and improves user visual comfort. Specifically, the system acquires ciliary muscle EMG signals in real time using non-contact EMG (electromyography) electrodes that are attached to the ciliary muscle's surface projection area while the device is worn. This reflects changes in muscle tension during lens accommodation, thereby inferring the eye's focusing intention. Simultaneously, the system samples the wearer's pupil diameter at a preset frequency using traditional visual sensors (such as near-infrared cameras), obtaining pupil diameter sampling data to assist in determining depth perception and light adaptation status. Through these multimodal physiological and behavioral signals (i.e., data on changes in eye state), the user's true focusing intention can be accurately depicted. After being timestamped and encoded, these heterogeneous data are input into a pre-trained optical parameter prediction module. For example, a multimodal AI (Artificial Intelligence) model, HybridNet, which integrates CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory) and Transformer structures, can collaboratively analyze spatial depth distribution, eye movement temporal patterns, and physiological response characteristics to predict target optical parameters that conform to the natural accommodation laws of the human eye, including the target radius of curvature required by the variable focus lenses (such as liquid lens arrays, variable focus liquid crystal lenses, metasurfaces, etc.) in head-mounted displays. The system dynamically adjusts the curvature of the variable focus lens accordingly, so that the optical imaging plane matches the depth of focus desired by the human eye in real time. This breaks through the VAC bottleneck caused by traditional fixed focal length design and manual focal length adjustment, and achieves hardware-level dynamic focusing response. It significantly improves the adaptability of the optical system in the head-mounted display to individual differences and dynamic scenes, and achieves more accurate, natural and low-latency focusing state adjustment. This fundamentally alleviates visual fatigue and dizziness caused by the mismatch between convergence and accommodation, and enhances the immersive experience and long-term wearing comfort of the head-mounted display. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating the first embodiment of the optical adjustment method of this application;

[0059] Figure 2 This is a flowchart illustrating the second embodiment of the optical adjustment method of this application.

[0060] Figure 3 This is a schematic diagram of the optical adjustment method in the embodiments of this application;

[0061] Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the optical adjustment method in the embodiments of this application.

[0062] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0063] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0064] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0065] With the rapid development of head-mounted display technology, optimizing the performance and user experience of head-mounted displays has become a key focus of industry research.

[0066] Traditional head-mounted displays (HMDs) primarily rely on fixed focal length designs or manual user adjustments for visual accommodation. This static adjustment mode offers limited freedom of choice, is heavily influenced by user subjective judgment, and struggles to adapt to dynamically changing visual scenarios and individual physiological differences. This often results in a discrepancy between the focusing state output by the optical system and the actual focusing needs of the human eye. This discrepancy exacerbates the VAC effect, manifesting as a lack of coordination between the convergence of the eyes and the accommodation of the lens when observing virtual objects through the HMD, leading to visual fatigue, dizziness, and other discomfort, severely impacting the user experience.

[0067] Therefore, how to more accurately adjust the focusing state of the optical system to reduce the VAC effect of the head-mounted display has become an urgent problem to be solved by those skilled in the art.

[0068] To address the aforementioned issues, this application provides an optical adjustment method applied to a head-mounted display device, which includes a variable focus lens. The method includes: acquiring data on changes in the wearer's eye state; predicting target optical parameters based on the eye state change data using a pre-trained optical parameter prediction model, wherein the target optical parameters include a target radius of curvature; and adjusting the curvature of the variable focus lens based on the target radius of curvature.

[0069] This application embodiment utilizes data on changes in the wearer's eye state, combined with a pre-trained optical parameter prediction model, to achieve dynamic and personalized adjustment of the optical system of the head-mounted display device. This effectively reduces the accommodative-verbose conflict (VAC effect) and improves user visual comfort. Specifically, the system acquires ciliary muscle EMG signals in real time using non-contact EMG (electromyography) electrodes that are attached to the ciliary muscle's surface projection area while the device is worn. This reflects changes in muscle tension during lens accommodation, thereby inferring the eye's focusing intention. Simultaneously, the system samples the wearer's pupil diameter at a preset frequency using traditional visual sensors (such as near-infrared cameras), obtaining pupil diameter sampling data to assist in determining depth perception and light adaptation status. Through these multimodal physiological and behavioral signals (i.e., data on changes in eye state), the user's true focusing intention can be accurately depicted. After being timestamped and encoded, these heterogeneous data are input into a pre-trained optical parameter prediction module. For example, a multimodal AI (Artificial Intelligence) model, HybridNet, which integrates CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory) and Transformer structures, can collaboratively analyze spatial depth distribution, eye movement temporal patterns, and physiological response characteristics to predict target optical parameters that conform to the natural accommodation laws of the human eye, including the target radius of curvature required by the variable focus lenses (such as liquid lens arrays, variable focus liquid crystal lenses, metasurfaces, etc.) in head-mounted displays. The system dynamically adjusts the curvature of the variable focus lens accordingly, so that the optical imaging plane matches the depth of focus desired by the human eye in real time. This breaks through the VAC bottleneck caused by traditional fixed focal length design and manual focal length adjustment, and achieves hardware-level dynamic focusing response. It significantly improves the adaptability of the optical system in the head-mounted display to individual differences and dynamic scenes, and achieves more accurate, natural and low-latency focusing state adjustment. This fundamentally alleviates visual fatigue and dizziness caused by the mismatch between convergence and accommodation, and enhances the immersive experience and long-term wearing comfort of the head-mounted display.

[0070] The head-mounted display device in this application embodiment may include, but is not limited to, head-mounted display devices such as Mixed Reality (MR) devices (e.g., MR glasses or MR helmets), Augmented Reality (AR) devices (e.g., AR glasses or AR helmets), Virtual Reality (VR) devices (e.g., VR glasses or VR helmets), Extended Reality (XR) devices, or some combination thereof. In this embodiment, for ease of description, the head-mounted display device will be used as the execution subject in the following description.

[0071] Based on this, embodiments of this application provide an optical adjustment method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the optical adjustment method of this application.

[0072] In this embodiment, the optical adjustment method is applied to a head-mounted display device, which includes a liquid lens array. The optical adjustment method includes steps S100 to S300:

[0073] Step S100: Obtain data on changes in the wearer's eye condition;

[0074] It should be noted that, in this embodiment, the technologies used in the head-mounted display device (i.e., head-mounted display device) mainly include AR, VR, MR, and XR technologies in a broader sense. Based on these technologies, when wearing the head-mounted display device, users can observe and interact with superimposed virtual content in a real or virtual environment through various sensors, cameras, or optical components in the head-mounted display device, thereby achieving an immersive visual experience.

[0075] To achieve accurate perception and dynamic response to the visual accommodation state of the human eye, this embodiment integrates a multimodal sensor in the head-mounted display device to collect data on changes in eye state. Upon detecting that the user is correctly wearing the head-mounted display device, the data acquisition function is automatically activated to collect key data closely related to the visual convergence-accommodation conflict (VAC). This multi-source collaborative sensing mechanism constitutes the data input foundation of the technical solution in this embodiment and is a prerequisite for achieving personalized, high-precision optical accommodation.

[0076] In this embodiment, eye state change data refers to multi-dimensional ocular biosignals that reflect the wearer's dynamic adjustment behavior and physiological response in visual tasks, mainly including eye-tracking event streams, pupil diameter sampling data, and ciliary muscle EMG signals.

[0077] Among them, eye-tracking event stream refers to an asynchronous event stream that continuously captures the subtle movements of the wearer's eyes at high resolution using event-based visual sensors. It accurately records fixation point switching, saccade trajectories, and instantaneous pupillary changes, and features low latency and high dynamic response. Binocular pupil diameter sampling data refers to data obtained by sampling the wearer's pupil diameter at a preset sampling frequency using traditional visual sensors. This data is used to quantify the pupil's response to depth of field, illumination, and cognitive load. Ciliary muscle EMG signals are acquired through non-invasive EMG electrodes integrated into the head-mounted display. When the head-mounted display is worn, these electrodes are in close contact with the wearer's ciliary muscle projection area on the wearer's body surface, enabling real-time acquisition of the muscle's electrical activity signals during lens accommodation, directly reflecting the eye's focusing intention.

[0078] This embodiment utilizes the aforementioned multimodal sensors to collaboratively collect information on the wearer's ocular physiological state, achieving precise characterization of the user's focusing intention. Compared to traditional methods that rely solely on fixed focal length design or manual focal length adjustment, this embodiment significantly improves the accuracy of recognizing actual focusing needs by using ocular state change data that reflects the user's internal eye adjustment intention, providing high-fidelity, low-latency data support for subsequent intelligent optical response.

[0079] Step S200: Based on the data on changes in eye state, the target optical parameters are predicted using a pre-trained optical parameter prediction model. The target optical parameters include the target radius of curvature.

[0080] In this embodiment, optical parameters refer to a set of adjustable parameters used to characterize the focusing state of the optical system in the head-mounted display device. These parameters directly determine the imaging quality, depth of focus, and visual comfort of the virtual image. Target optical parameters refer to a set of ideal optical parameter values ​​calculated by the optical parameter prediction model based on model inputs (including data on changes in eye state) to optimize the current visual experience. These target optical parameters represent the target state of system adjustment, aiming to precisely match the output of the optical system (such as the position of the imaging plane) with the natural accommodation needs of the human eye, thereby minimizing the visual convergence-accommodation conflict (VAC).

[0081] In this embodiment, the target optical parameters include at least the target radius of curvature. This target radius of curvature comprises the desired radius of curvature value of the adjustable refractive surface (hereinafter referred to as the adjustable refractive surface) in each micro-liquid lens of the liquid lens array, and is one of the core parameters in the target optical parameters. This target radius of curvature is derived by the optical parameter prediction model based on the input multimodal data, and directly corresponds to the depth plane that the human eye expects to focus on under the current visual task. By adjusting the actual radius of curvature of the adjustable refractive surface of each micro-liquid lens in the liquid lens array to match the target radius of curvature, real-time, continuous, and precise movement of the optical focusing plane can be achieved, thereby providing a dynamic visual experience synchronized with the human eye's accommodation.

[0082] It should be noted that, in this embodiment, the optical parameter prediction model is a multimodal deep learning model trained on a large number of optical parameter prediction samples. For example, this optical parameter prediction model can be based on a HybridNet model that integrates CNN, LSTM, and Transformer architectures, trained on a large number of optical parameter prediction samples.

[0083] In one feasible implementation, step S200 above may include step S210:

[0084] Step S210: Input the eye state change data into a pre-trained optical parameter prediction model to predict the target optical parameters.

[0085] In this embodiment, the optical parameter prediction samples used when training the optical parameter prediction model use eye state change data as sample features and the optimal optical parameters actually measured or simulated under the corresponding conditions as sample labels. Therefore, eye state change data can be used as the model input of the optical parameter prediction model to obtain the target optical parameters predicted by the optical parameter prediction model.

[0086] In another feasible implementation, step S200 above may further include steps S220 to S230:

[0087] Step S220: Obtain the wearer's environmental perception depth data;

[0088] In this embodiment, environmental perception depth data refers to the spatial structure information of the three-dimensional scene in the wearer's current field of vision. Specifically, it can be a scene depth map collected by a depth sensor (such as a time-of-flight camera), or scene depth information extracted from the light spot image of the wearer's eyes. Both the scene depth map and the scene depth information contain distance information between each object and the wearer's eyes from the wearer's perspective, dynamically representing the depth distribution of target objects in virtual or real environments, and serving as a key input for determining external visual needs.

[0089] Step S230: Input the eye state change data and the environmental perception depth data into the pre-trained optical parameter prediction model to predict the target optical parameters.

[0090] In this embodiment, the optical parameter prediction samples used when training the optical parameter prediction model use eye state change data and environmental perception depth data as sample features, and the optimal optical parameters actually measured or simulated under the corresponding conditions as sample labels. Therefore, eye state change data and environmental perception depth data can be used as model inputs for the optical parameter prediction model to obtain the target optical parameters predicted by the optical parameter prediction model.

[0091] Compared to optical parameter prediction samples that rely solely on changes in eye state, introducing environmentally perceived depth data as an additional input feature enhances the optical parameter prediction model's understanding of the external visual scene. This allows the prediction of target optical parameters to be based not only on the user's physiological intent but also on the spatial structure information of the actual external environment, thereby improving the accuracy and robustness of the prediction results. Especially when the user's gaze intent is blurred or the signal-to-noise ratio of the eye signal is low, environmentally perceived depth data can serve as an important prior constraint, assisting the optical parameter prediction model in inferring a target focusing depth that better matches real visual needs. Furthermore, by fusing the user's internal adjustment intent with the depth distribution of the external scene, the optical parameter prediction model can effectively distinguish between different visual behavior modes such as "active focusing" and "passive adaptation," further optimizing personalized optical response strategies and achieving more natural and precise dynamic focusing adjustment.

[0092] In this embodiment, the acquired multi-source data, including eye-tracking event streams, pupil diameter sampling data, ciliary muscle EMG signals, and scene depth maps, undergoes time-stamp alignment and feature scaling preprocessing to ensure strict temporal synchronization and dimensional consistency across the modal data. Subsequently, this data is jointly input into a pre-trained optical parameter prediction model, which predicts end-to-end target optical parameters conforming to the natural accommodation patterns of the human eye, including key parameters such as the target radius of curvature and focal length required for a variable-focus lens. This prediction process fully considers individual physiological differences and dynamic scene changes, achieving intelligent mapping from "perceived intent" to "optical response."

[0093] Step S300: Adjust the curvature of the variable focus lens based on the target radius of curvature.

[0094] It should be noted that, in this embodiment, a variable focus lens refers to an optical element whose optical curvature can be dynamically adjusted by external control signals (such as voltage, current, mechanical stress, etc.). It can achieve continuous changes in focal length without altering its physical position, thereby adapting to the imaging needs of different depth planes. Variable focus lenses feature fast response speed, high adjustment precision, and compact size, making them suitable for applications in head-mounted displays that require high real-time performance and miniaturization.

[0095] For example, the variable focus lens can be a liquid lens array, a variable focus liquid crystal lens, a metasurface tunable optical element, etc.

[0096] Among them, liquid lens array refers to an array composed of miniature liquid lenses. As those skilled in the art know, a liquid lens is a dynamic optical element that can change the curvature of a liquid interface by applying an external voltage. Its curvature adjustment is based on the principle of electrowetting or dielectric elastomer, with fast response speed (millisecond level), continuous focusing capability, and is suitable for real-time optical compensation.

[0097] A zoomable liquid crystal lens is an optical device that uses the orientation change of liquid crystal molecules under the influence of an electric field to alter its equivalent refractive index distribution and thus adjust the lens focal length. Its structure typically includes two layers of transparent electrodes, a nematic liquid crystal material filling the space between them, and a patterned electrode layer for forming a non-uniform electric field. By adjusting the magnitude and distribution of the applied voltage, precise control of the equivalent radius of curvature of the liquid crystal lens can be achieved. It offers advantages such as low power consumption, no moving mechanical parts, and ease of integration.

[0098] Metasurface tunable optical elements are planar optical devices designed based on subwavelength-scale nanostructure arrays (i.e., metasurfaces). By introducing tunable materials (such as phase change materials, liquid crystals, or microelectromechanical systems) into the metasurface unit structure, their local phase response characteristics can be changed under external field excitation, thereby achieving dynamic control of the incident light wavefront. These elements can achieve ultra-thin, lightweight, and high-speed focusing, representing the development direction of next-generation variable-focus optical devices.

[0099] In this embodiment, taking a liquid lens array as an example, when the variable focus lens is a liquid lens array, the head-mounted display device can calculate the required driving voltage according to the target radius of curvature output in step S200, and apply it to the liquid lens array, thereby precisely controlling its lens curvature to the target value.

[0100] The relationship between the radius of curvature R of the liquid lens and the driving voltage V is as follows:

[0101] ;

[0102] in:

[0103] R: Target radius of curvature (μm).

[0104] V: Drive voltage (V), the target value to be calculated.

[0105] k: Lens material property constant (e.g., for dielectric elastomers, k = 0.12 μm / V²).

[0106] Vacuum dielectric constant (8.85e-12 F / m).

[0107] The relative permittivity of the material (e.g., PDMS). =2.8).

[0108] d: Electrode spacing (μm, default 50μm).

[0109] R0: Zero voltage radius of curvature (determined by the initial shape of the lens, typically 200 μm).

[0110] This process enables the optical imaging plane to match the depth of focus desired by the human eye in real time, effectively eliminating the visual convergence-accommodation conflict (VAC) caused by fixed focal length design or inaccurate manual focusing, thereby improving image clarity and visual comfort.

[0111] Furthermore, the mapping relationship between the curvature radius of the liquid lens array and the driving voltage can be pre-calibrated, allowing the driving voltage corresponding to the target curvature radius to be determined by querying this mapping relationship. Specifically, the calibration process can be completed before the device leaves the factory. By applying different voltages under a standard test environment and measuring the corresponding actual curvature radius, a driving voltage-curvature radius lookup table or a fitted nonlinear function model is established and stored in the local storage unit of the head-mounted display device. During operation, the system quickly obtains the required driving voltage based on the predicted target curvature radius through interpolation or analytical calculation, and the driving circuit applies it to the liquid lens array to achieve closed-loop control.

[0112] Those skilled in the art already have some understanding of curvature adjustment for other variable focus lenses, such as variable focus liquid crystal lenses and metasurface adjustable optical elements, and this embodiment will not elaborate on them here.

[0113] This embodiment integrates multimodal ocular physiological signals with environmental depth perception, combined with AI-driven optical parameter prediction and liquid lens dynamic focusing, to construct a closed-loop, adaptive intelligent optical adjustment system. This solution overcomes the limitations of traditional head-mounted display optical design, achieving accurate recognition of the user's true focusing intention and rapid hardware-level response. It significantly reduces visual fatigue and dizziness caused by the VAC effect, and improves the immersion, usability, and long-term wearing comfort of the head-mounted display.

[0114] This embodiment integrates the wearer's eye state change data with environmental perception depth data, combined with a pre-trained optical parameter prediction model, to achieve dynamic and personalized adjustment of the head-mounted display's optical system. This effectively reduces visual convergence-accommodation conflict and improves user visual comfort. Specifically, the system utilizes event-based visual sensors to collect dynamic information from the wearer's eyes at high temporal resolution, generating an eye-tracking event stream to accurately capture rapid eye movements such as gaze shifts, saccades, and pupil dilation / contraction. Simultaneously, the system uses non-contact EMG (Electromyography) electrodes, which are attached to the ciliary muscle's surface projection area while the device is worn, to acquire ciliary muscle EMG signals in real time. This reflects changes in muscle tension during lens accommodation, allowing the system to infer the eye's focusing intent. Furthermore, the system uses traditional visual sensors (such as near-infrared cameras) to sample the wearer's pupil diameter at a preset frequency, obtaining pupil diameter sampling data to assist in determining depth perception and illumination adaptation status. The aforementioned multimodal physiological and behavioral signals (i.e., data on changes in eye state), combined with the scene depth map (i.e., environmental perception depth data, containing depth information of various objects in space from the wearer's current perspective) obtained by depth sensors (such as time-of-flight cameras), together constitute a comprehensive characterization of the user's true focusing intention. After timestamp alignment and feature encoding, this heterogeneous data is input into a pre-trained optical parameter prediction module, such as a fusion of CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory) and Transformer structures. This module can collaboratively analyze the spatial depth distribution, eye movement temporal patterns, and physiological response characteristics of the multimodal AI (Artificial Intelligence) model HybridNet, thereby predicting target optical parameters that conform to the natural accommodation laws of the human eye, including the target radius of curvature required by the liquid lens array in the head-mounted display device. The system dynamically adjusts the curvature of the liquid lens accordingly, so that the optical imaging plane matches the depth of focus desired by the human eye in real time. This breaks through the VAC bottleneck caused by traditional fixed focal length design and manual focal length adjustment, and achieves hardware-level dynamic focusing response. It significantly improves the adaptability of the optical system in the head-mounted display to individual differences and dynamic scenes, and realizes more accurate, natural and low-latency focusing state adjustment. It fundamentally alleviates visual fatigue and dizziness caused by the mismatch between convergence and accommodation, and enhances the immersive experience and long-term wearing comfort of the head-mounted display.

[0115] In one feasible implementation, the target optical parameters include the target lens focal length, and before step S300 above, the optical adjustment method may further include steps A10~A20:

[0116] Step A10: Obtain the refractive index of the variable focal length lens material, and verify the focal length and radius of curvature of the target lens using the thin lens focal length formula based on the material refractive index to obtain the verification result of the thin lens focal length formula.

[0117] It should be noted that, in this embodiment, the target optical parameters also include the target lens focal length. This target lens focal length is the desired focal length value of the variable focal length lens and is one of the core parameters in the target optical parameters. This target lens focal length is inferred by the optical parameter prediction model based on the input multimodal data and directly corresponds to the depth plane that the human eye expects to focus on under the current visual task. The target lens focal length, together with the target radius of curvature, constitutes a complete description of the physical state of the variable focal length lens.

[0118] Since the optical behavior of a variable focal length lens follows basic geometric optical laws, there is a theoretical functional relationship between its focal length f and its radius of curvature R, namely the focal length formula for a thin lens:

[0119] ;

[0120] in:

[0121] n: The refractive index of the filler material of the variable focus lens (e.g., polydimethylsiloxane, n=1.43).

[0122] R: The radius of curvature of the refractive surface on which the light is incident in the two refractive surfaces of the zoom lens;

[0123] R fix The radius of curvature of the refractive surface on which light rays emerge from the two refractive surfaces of a zoom lens;

[0124] In one example, preferably, in a variable focal length lens, the refracting surface on which the light is incident is an adjustable refracting surface, and the refracting surface on which the light is exited is a fixed refracting surface (i.e., a refracting surface whose curvature is not adjustable). The above step A10 is: to obtain the material refractive index of the variable focal length lens and the radius of curvature of the fixed refracting surface, and to perform a thin lens focal length formula verification on the target lens focal length and the target radius of curvature based on the material refractive index and the radius of curvature of the fixed refracting surface, so as to obtain the thin lens focal length formula verification result.

[0125] In practical applications, there may be a discrepancy between the target lens focal length predicted by the optical parameter model and the theoretical focal length calculated based on the target radius of curvature. If the focusing operation is performed directly based on the target radius of curvature (i.e., step S300), it may introduce imaging deviation or aggravate the VAC effect.

[0126] To ensure the physical rationality of optical adjustment and system stability, this implementation introduces a thin lens focal length formula verification mechanism.

[0127] For example, the head-mounted display device first obtains the refractive index (i.e., material refractive index) of the material used in the variable-focus lens. This material refractive index can be obtained through factory calibration data, material database queries, or real-time monitoring by a built-in miniature refractive index sensor. Subsequently, the system compares the target lens focal length output in step S200 with the theoretical focal length calculated based on the target radius of curvature and the material refractive index to determine whether the focal length error between the two is within a preset tolerance range. If the focal length error is within the preset tolerance range, it is determined as "verification passed"; otherwise, it is determined as "verification failed".

[0128] Step A20: If the verification result of the thin lens focal length formula is passed, perform the step of adjusting the curvature of the variable focal length lens based on the target radius of curvature.

[0129] In this embodiment, the system only allows the execution of step S300, which adjusts the curvature of the variable focus lens based on the target radius of curvature, to complete the focusing state adjustment of the optical system, if the verification result is "verification passed". If the verification result is "verification failed", the system can trigger an exception handling mechanism, such as discarding the current prediction result, starting a model self-check, re-acquiring data for prediction, or switching to a backup focusing strategy, and can issue a prompt to the user.

[0130] This implementation method, based on the first embodiment, introduces an optical parameter consistency verification mechanism based on physical laws. Its technical contributions are mainly reflected in the following two aspects:

[0131] Enhancing system reliability and safety: By introducing the physical constraint of the thin lens focal length formula, the "soft decision" output by the optical parameter prediction model is verified by "hard rules". This effectively prevents unreasonable optical parameters caused by model prediction errors, data anomalies or system noise from being directly executed, avoiding the risk of image blurring, ghosting or increased visual discomfort caused by incorrect focusing, and significantly enhancing the robustness and safety of the entire optical adjustment system.

[0132] Ensuring the physical accuracy of optical adjustment: This verification mechanism ensures that the mapping relationship from the "target radius of curvature" to the actual imaging focal length strictly conforms to the laws of optical physics, so that the prediction results of the optical parameter prediction model are consistent with the physical response of the hardware, thereby improving the accuracy and reliability of focusing actions and avoiding the disconnect between "intelligent prediction" and "physical reality".

[0133] In summary, this implementation method is not a simple functional aggregation, but rather achieves a deep integration of intelligence and physical determinism by constructing a verification layer based on physical laws between "model prediction" and "hardware execution." This design significantly improves the engineering practicality of the optical adjustment system and represents a crucial step from "theoretically feasible" to "engineering reliable" in this application's technical solution. It reflects a profound consideration of the core requirements of security, accuracy, and stability in complex human-computer interaction systems, constituting a significant technical contribution that distinguishes it from existing technologies.

[0134] In one feasible implementation, after step S210, the optical adjustment method may further include steps B10 to B20:

[0135] Step B10: Input the target radius of curvature into the pre-trained distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model;

[0136] In this embodiment, distortion parameters refer to a set of adjustable parameters used for pre-distortion in the rendering pipeline of the head-mounted display device. These parameters directly determine the degree of geometric deformation compensation presented by the virtual image under the action of a dynamic optical system (such as a liquid lens array). Target distortion parameters refer to the ideal distortion parameter values ​​calculated by the distortion parameter prediction model based on the model input (the target radius of curvature of the liquid lens), used to optimize the geometric fidelity and visual consistency of the current image. These target distortion parameters are the core parameters for achieving distortion correction, aiming to ensure that the pre-distorted image generated during the rendering stage, after propagation through the current optical system, is accurately restored to geometrically distortion-free visual content. This ensures that the optical imaging results match the natural visual perception of the human eye, effectively eliminating visual artifacts such as image distortion and edge stretching caused by dynamic focusing, and improving the realism of the imaging and the comfort of the user experience.

[0137] In this embodiment, the distortion parameter prediction model is a deep learning model trained on a large number of distortion parameter prediction samples. For example, this distortion parameter prediction model can be based on the DistNet (Distortion Correction Network) model, trained on a large number of distortion parameter prediction samples. The DistNet model may include four CNN layers (for extracting local optical features caused by curvature changes) and two LSTM layers (for modeling the temporal dynamics of gaze behavior), and finally outputs the predicted radial distortion coefficients k1 and k2 as target distortion parameters through a regression layer.

[0138] It is easy to understand that, depending on actual needs, this distortion parameter prediction model can also be set to output other distortion coefficients besides the radial distortion coefficient, such as the tangential distortion coefficient, thin prism distortion term, or higher-order polynomial distortion parameters. This can be achieved simply by adjusting the sample labels in the distortion parameter prediction samples.

[0139] In this embodiment, the distortion parameter prediction samples used when training the distortion parameter prediction model are characterized by the lens curvature of the liquid lens array, and the optimal distortion parameters actually measured or simulated under the corresponding conditions are used as sample labels. Therefore, the target radius of curvature can be used as the model input of the distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model.

[0140] In this embodiment, the target radius of curvature predicted by the optical parameter prediction model is input into a pre-trained distortion parameter prediction model. The distortion parameter prediction model then predicts the target distortion parameters that match the current optical configuration end-to-end. This prediction process considers the optical deformation characteristics caused by dynamic focusing and fully takes into account the nonlinear changes in the distortion field under different lens curvature states. It achieves intelligent mapping from "optical state perception" to "image pre-distortion parameter generation," providing a high-precision, low-latency control basis for real-time distortion correction in the subsequent rendering pipeline.

[0141] Step B20: Inject the target distortion parameters into the rendering pipeline of the head-mounted display device.

[0142] As those skilled in the art will know, the rendering pipeline refers to the graphics processing flow in a head-mounted display device responsible for generating the final displayed image, typically including stages such as vertex shading, rasterization, fragment shading, and pre-distortion. Among these, to counteract image distortion introduced by the dynamic optical system (liquid lens), "inverse distortion" (i.e., pre-distortion) is generally applied in advance during the rendering stage to restore the image after propagation through the optical system to a distortion-free state.

[0143] This embodiment considers that after curvature adjustment, the original distortion parameters of the liquid lens array may no longer be applicable. Therefore, it is necessary to recalculate the target distortion parameters based on the latest focusing adjustment state of the optical system to ensure that, regardless of the curvature state of the liquid lens array after curvature adjustment, the image seen by the user always maintains geometric accuracy and visual naturalness. Thus, after determining the target distortion parameters using a pre-trained distortion parameter prediction model, this embodiment also needs to inject them into the rendering pipeline of the head-mounted display device to replace the original distortion parameters and make them effective.

[0144] It is worth mentioning that while performing step B20 above, the depth-of-field blur radius in the rendering relationship also needs to be adjusted simultaneously to match the focal length of the target lens. This ensures that after the image geometry changes (i.e., distortion correction), visual effects such as depth-of-field blur can still accurately and naturally match the new imaging space, avoid visual artifacts, and maintain the realism and consistency of the rendering.

[0145] Based on the "dynamic focusing" of the first embodiment mentioned above, this implementation further introduces a "real-time distortion prediction and rendering compensation" mechanism, constructing a three-in-one full-link optical optimization system of "sensing-adjustment-compensation". Its technical contributions are reflected in the following three aspects:

[0146] Solving the image fidelity challenge in dynamic optical systems: Traditional head-mounted displays often rely on fixed optical models for distortion correction, which cannot adapt to time-varying distortion caused by continuous changes in the curvature of liquid lenses. This embodiment achieves adaptive distortion compensation by predicting the target distortion parameters under the current optical state in real time and dynamically injecting them into the rendering pipeline. This effectively avoids image distortion, edge stretching, or text deformation caused by focusing, significantly improving visual realism and information readability during dynamic focusing.

[0147] A closed-loop architecture for collaborative optimization of "optics-rendering": This solution is the first to use the physical optical state (lens curvature) as the input for distortion prediction, breaking through the traditional static compensation mode that only relies on the calibration model, so that distortion correction matches the real state of optical components, ensuring that high-precision correction can still be maintained in scenes with a large field of view.

[0148] Improving overall system response efficiency and user experience: By immediately initiating the distortion prediction and correction process after curvature adjustment or thin lens focal length formula verification, the system can complete parameter preparation before image rendering, avoiding delays or stuttering. Combined with an AI-driven distortion parameter prediction model, the entire distortion correction process has extremely low latency (milliseconds), achieving a seamless "focusing and correction" experience, fundamentally solving the technical bottleneck of "clear vision but distortion" in dynamic optical systems.

[0149] In summary, this implementation method is not an isolated functional module, but a necessary extension and closed-loop improvement of the preceding optical adjustment actions. It deeply integrates hardware-level optical adjustment with software-level image rendering, constructing a truly "intelligent optics-graphics collaborative system," which significantly improves the imaging quality and user experience of the head-mounted display device in dynamic focusing scenarios.

[0150] In one feasible implementation, after step S230 described above, the optical adjustment method may further include steps C10 to C30:

[0151] Step C10: Obtain the wearer's gaze information, which includes gaze point coordinates, gaze depth, and convergence angle;

[0152] It should be noted that, in this embodiment, gaze information refers to the visual behavior and spatial geometric features formed when the wearer's eyes work together to focus on a specific target under the current visual task, specifically including:

[0153] Gaze coordinates: refers to the two-dimensional spatial position of the wearer's current gaze point in the display coordinate system of the head-mounted display device, such as normalized screen coordinates (x, y), which are usually calculated in real time by the eye-tracking system;

[0154] Depth of gaze: refers to the straight-line distance (in meters) of the gaze point relative to the human eye in three-dimensional space. It can be extracted from the depth value corresponding to the coordinate position of the gaze point in environmental perception depth data (such as scene depth map), or calculated backward by combining the spatial geometric relationship between the binocular optical center and the gaze point.

[0155] Convergence angle: The angle (unit: degrees or radians) formed when the visual axes of both eyes focus on the same target (i.e., the fixation point). Its magnitude is inversely proportional to the depth of fixation. This angle can be calculated from the position of the center of the pupils of both eyes, the coordinates of the fixation point, and the depth of fixation. It is a core physiological indicator for judging the state of binocular co-focusing.

[0156] Step C20: Input the target radius of curvature and gaze information into the pre-trained distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model;

[0157] In this embodiment, the distortion parameter prediction samples used when training the distortion parameter prediction model use the lens curvature of the liquid lens array and the wearer's gaze information (including gaze point coordinates, gaze depth, convergence angle, etc.) as sample features, and the optimal distortion parameters actually measured or simulated under corresponding conditions as sample labels. Therefore, the target radius of curvature and gaze information can be used as model inputs to the distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model.

[0158] Compared to distortion parameter prediction samples that rely solely on the lens curvature of the liquid lens array as a sample feature, incorporating the wearer's gaze information as an additional input feature significantly enhances the spatial sensitivity and personalized adaptation capabilities of distortion parameter prediction. Specifically, optical distortion caused by the liquid lens array under different curvature states is non-uniform and viewpoint-dependent, especially in off-center gaze regions (such as the edge of the field of view), where local stretching, compression, or trapezoidal distortion can easily occur. Relying solely on the overall lens curvature makes it difficult to accurately model such spatially heterogeneous distortion fields. However, by introducing user gaze behavior information such as gaze point coordinates and convergence angle, the distortion parameter prediction model can perceive the user's current visual focus area and its spatial geometric relationship, thereby dynamically predicting local distortion compensation parameters that match the gaze area, achieving "on-demand correction" rather than "global adjustment." Furthermore, by combining gaze depth information and injecting the target distortion parameters into the head-mounted display's rendering pipeline model, it can also predict the impact of depth-of-field plane migration caused by focusing actions on the distortion distribution in the surrounding area, further optimizing the pre-distortion strategy and improving correction accuracy and visual coherence.

[0159] In this embodiment, the multi-dimensional input data obtained in step C10, including the wearer's gaze coordinates, gaze depth, and convergence angle, as well as the target curvature radius predicted by the optical parameter prediction model, are first preprocessed with timestamp alignment and normalization to ensure strict temporal synchronization and dimensional consistency between optical state information and visual behavior data. Subsequently, this data is jointly input into a pre-trained distortion parameter prediction model, which predicts the target distortion parameters that match the current optical configuration and visual intent end-to-end. This prediction process deeply integrates the spatial geometric relationship between the optical deformation characteristics caused by dynamic focusing and the user's gaze behavior, fully considering the nonlinear changes in the distortion field under different lens curvature states and the local distortion differences caused by eccentric gaze. It achieves intelligent mapping from collaborative perception of "optical state - visual intent" to "image pre-distortion parameter generation," providing a high-precision, low-latency control basis for real-time distortion correction in the subsequent rendering pipeline.

[0160] Step C30: Inject the target distortion parameters into the rendering pipeline of the head-mounted display device.

[0161] This embodiment considers that after curvature adjustment, the original distortion parameters of the liquid lens array may no longer be applicable. Therefore, it is necessary to recalculate the target distortion parameters based on the latest focusing adjustment state of the optical system to ensure that, regardless of the curvature state of the liquid lens array after curvature adjustment, the image seen by the user always maintains geometric accuracy and visual naturalness. Thus, after determining the target distortion parameters using a pre-trained distortion parameter prediction model, this embodiment also needs to inject them into the rendering pipeline of the head-mounted display device to replace the original distortion parameters and make them effective.

[0162] It is worth mentioning that while performing step C30 above, the depth-of-field blur radius in the rendering relationship also needs to be adjusted simultaneously to match the focal length of the target lens. This ensures that after the image geometry changes (i.e., distortion correction), visual effects such as depth-of-field blur can still accurately and naturally match the new imaging space, avoid visual artifacts, and maintain the realism and consistency of the rendering.

[0163] Based on the "dynamic focusing" and "light field shaping" of the first embodiment, this implementation further introduces a "real-time distortion prediction and rendering compensation" mechanism, constructing a three-in-one full-link optical optimization system of "sensing-adjustment-compensation". Its technical contributions are reflected in the following three aspects:

[0164] Solving the image fidelity challenge in dynamic optical systems: Traditional head-mounted displays often rely on fixed optical models for distortion correction, which cannot adapt to time-varying distortion caused by continuous changes in the curvature of liquid lenses. This embodiment achieves adaptive distortion compensation by predicting the target distortion parameters under the current optical state in real time and dynamically injecting them into the rendering pipeline. This effectively avoids image distortion, edge stretching, or text deformation caused by focusing, significantly improving visual realism and information readability during dynamic focusing.

[0165] A closed-loop architecture for collaborative optimization of "optics-rendering": This solution is the first to use both physical optical state (lens curvature) and user visual behavior (gaze information) as inputs for distortion prediction, breaking through the traditional static compensation mode that relies solely on calibration models. This collaborative perception mechanism of "human-machine-environment" enables distortion correction to not only match the hardware state but also match the user's actual viewing behavior, maintaining high-precision correction even in scenarios with eccentric gaze or large field of view.

[0166] Improving overall system response efficiency and user experience: By immediately initiating the distortion prediction and correction process after curvature adjustment or formula verification, the system can complete parameter preparation before image rendering, avoiding delays or stuttering. Combined with an AI-driven distortion parameter prediction model, the entire distortion correction process has extremely low latency (milliseconds), achieving a seamless "focusing and correction" experience, fundamentally solving the technical bottleneck of "clear vision but distortion" in dynamic optical systems.

[0167] In summary, this implementation method is not an isolated functional module, but a necessary extension and closed-loop improvement of the preceding optical adjustment actions. It deeply integrates hardware-level optical adjustment with software-level image rendering, constructing a truly "intelligent optics-graphics collaborative system," which significantly improves the imaging quality and user experience of the head-mounted display device in dynamic focusing scenarios.

[0168] It is worth mentioning that, in the above embodiments and implementations of this application, both the optical parameter prediction model and the distortion parameter prediction model can be pre-trained using a large number of pre-prepared samples in the offline stage, and dynamically fine-tuned in the online stage by combining user feedback and rendering quality through a reinforcement learning framework (such as a near-end policy optimization algorithm) to continuously optimize the prediction accuracy.

[0169] For example, in one feasible implementation, after step S300 described above, the optical adjustment method may further include steps D10 to D30:

[0170] Step D10: Obtain the wearer's comfort score and calculate the rendering quality index of the head-mounted display device;

[0171] It should be noted that, in this embodiment, the comfort score refers to a quantitative indicator reflecting the quality of the wearer's visual experience while using the head-mounted display device. This comfort score can be obtained in various ways, and its form is not limited, aiming to flexibly adapt to different device configurations and user preferences.

[0172] Specifically, comfort scores can be obtained through any of the following methods or a combination thereof:

[0173] Subjective rating based on active input: The system obtains a second comfort rating (e.g., 1-5 points) submitted by the wearer through an active rating module integrated into the head-mounted display (such as a touch bar, voice recognition interface, or gesture recognition system). This second comfort rating directly reflects the user's subjective feelings about the current image clarity, focus matching, and visual comfort. This method is simple to implement, provides clear feedback, and is suitable for scenarios where users are willing to participate in the interaction.

[0174] Objective scoring based on physiological signals: The system acquires the wearer's ciliary muscle EMG data and calculates the ciliary muscle tension, which reflects the degree of tension of the eye's accommodation muscles, as characterized by the level of electromyographic activity. Simultaneously, it acquires pupil diameter sampling data and calculates its rate of change, which is closely related to visual fatigue and dizziness. These physiological indicators are input into a preset automatic scoring module (which can use a lookup table, empirical formula, or lightweight neural network), outputting a first comfort score that maps the ciliary muscle tension to the rate of change. This method requires no active user intervention, enabling seamless and continuous monitoring, and is suitable for prolonged wear or immersive scenarios.

[0175] For example, in one feasible implementation, the head-mounted display device includes an automatic scoring module, and the step of obtaining the wearer's comfort score in step D10 above may include steps D11 to D14:

[0176] Step D11: Obtain the ciliary muscle EMG data of the wearer, and calculate the ciliary muscle tension of the wearer based on the ciliary muscle EMG data;

[0177] Step D12: Obtain the wearer's pupil diameter sampling data, and calculate the wearer's pupil diameter change rate based on the pupil diameter sampling data;

[0178] Step D13: Input the ciliary muscle tension and pupil diameter change rate into the automatic scoring module to obtain the first comfort score output by the automatic scoring module;

[0179] Step D14: Determine the wearer's comfort score based on the first comfort score.

[0180] Subjective and objective rating fusion: The system simultaneously acquires the aforementioned subjective ratings (i.e., the second comfort rating) and objective ratings (i.e., the first comfort rating), and generates the final comfort rating through a weighted fusion strategy (such as linear weighting, dynamic weight adjustment, or a confidence-based selection mechanism). This approach combines the accuracy of subjective intent with the continuity of physiological signals, thereby improving the robustness and reliability of the feedback signals.

[0181] For example, step D14 above may include steps D15-D16:

[0182] Step D15: Obtain the second comfort score submitted by the wearer through the active rating module;

[0183] Step D16: Weighted fusion of the first comfort score and the second comfort score to obtain the wearer's comfort score.

[0184] It should also be noted that, in this embodiment, rendering quality metrics refer to technical parameters used to objectively evaluate the rendering quality of virtual images, typically including:

[0185] PSNR (Peak Signal-to-Noise Ratio): Measures the pixel-level difference between an image after pre-distortion compensation and an ideal distortion-free image;

[0186] SSIM (Structural Similarity Index Measure): Evaluates the fidelity of an image in terms of brightness, contrast, and structural information;

[0187] ESG (Edge Sharpness Gradient): Reflects the sharpness of the edges of text or outlines.

[0188] Step D20: Input the comfort score and rendering quality index into the preset first reward function to obtain the first reward value output by the first reward function;

[0189] As those skilled in the art will recognize, a reward function is the core evaluation mechanism in a reinforcement learning framework. It is used to synthesize multi-source feedback signals into a scalarized reward signal, or reward value. This reward value serves as input to the reinforcement learning algorithm, evaluating the merits of the current strategy or action: a larger reward value indicates that the current strategy or action is more conducive to achieving the goal, while a smaller reward value indicates that the current strategy or action is more likely to lead to performance degradation. By maximizing the accumulated reward value, the agent (such as the AI ​​model in this application) can gradually learn the optimal decision-making strategy. The design of the reward function directly affects learning efficiency and convergence direction, serving as a crucial bridge connecting the system's objective and the model's behavior.

[0190] In this embodiment, the first reward function is a reinforcement learning evaluation function specifically used to assess the quality of curvature adjustment decisions generated by the optical parameter prediction model. The first reward value, output by the first reward function, quantifies the overall performance of the current curvature adjustment action. This first reward value serves as a feedback signal for the reinforcement learning algorithm, driving the update of the weight parameters of the optical parameter prediction model, enabling online optimization and personalized adaptation of the model during real-world use. A higher first reward value indicates a better current curvature adjustment strategy, and the system will tend to repeat such behavior.

[0191] In this embodiment, the first reward function can also be set to generate a first reward value based on the comfort score, rendering instruction indicators, and system latency. Here, system latency refers to the time required for the system to complete one cycle of steps S100 to S300, that is, the time taken from acquiring data on changes in the wearer's eye state to adjusting the curvature of the variable focus lens. This system latency should be as small as possible compared to the screen refresh time interval.

[0192] Step D30: Update the weight parameters of the optical parameter prediction model based on the first reward value.

[0193] This implementation method uses a first reward value and reinforcement learning algorithms such as near-end policy optimization to update the weight parameters of the optical parameter prediction model using gradients. This process enables the optical parameter prediction model to continuously learn in real-world scenarios, gradually optimizing its mapping capability from multimodal inputs to target optical parameters, thereby achieving personalized and adaptive focus control.

[0194] In another feasible implementation, after step B20 or step C30 described above, the optical adjustment method may further include:

[0195] Step E10: Obtain the wearer's comfort score and calculate the rendering quality index of the head-mounted display device;

[0196] Step E20: Input the comfort score and rendering quality index into the preset second reward function to obtain the second reward value output by the second reward function;

[0197] In this embodiment, the second reward function is a reinforcement learning evaluation function specifically used to assess the quality of distortion correction decisions generated by the distortion parameter prediction model. The second reward value, output by the second reward function, quantifies the overall performance of the current distortion correction action. This second reward value serves as a feedback signal for the reinforcement learning algorithm, driving the update of the weight parameters of the distortion parameter prediction model, enabling online optimization and personalized adaptation of the model during real-world use. A higher second reward value indicates a better current distortion correction strategy, and the system will tend to repeat such behavior.

[0198] In this embodiment, the second reward function can also be set to generate a second reward value based on the comfort score, rendering instruction indicators, and system latency.

[0199] Step E30: Update the weight parameters of the distortion parameter prediction model based on the second reward value.

[0200] This implementation method, based on a second reward value, employs reinforcement learning algorithms such as proximal policy optimization to update the weight parameters of the distortion parameter prediction model using gradients. This process enables the distortion parameter prediction model to continuously learn in real-world scenarios, gradually optimizing its mapping capability from multimodal inputs to target distortion parameters, thereby achieving personalized and adaptive focus control.

[0201] Based on the first embodiment of this application, an optical adjustment method according to the second embodiment of this application is proposed, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the optical adjustment method of this application.

[0202] In the second embodiment of this application, the same or similar content as in the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0203] In this embodiment, the head-mounted display device further includes a photosensitive sensor array and a movable grating. After step S300, the optical adjustment method may further include steps S400 to S500:

[0204] Step S400: Obtain the light intensity distribution data of the display screen of the head-mounted display device, and calculate the current light field uniformity information of the head-mounted display device based on the light intensity distribution data;

[0205] As those skilled in the art will know, illuminance refers to the luminous flux received per unit area, typically measured in lux or candela per square meter, and is used to quantify the brightness of light at a given spatial location. In head-mounted displays, illuminance reflects the energy distribution level of the virtual image on the user's retinal imaging plane, directly affecting the perceived brightness and clarity of vision.

[0206] Light field uniformity refers to the consistency of light intensity distribution at different spatial locations within the field of view of a head-mounted display device, i.e., the spatial balance of light energy. High light field uniformity means that the overall brightness distribution of the displayed image is smooth with no significant differences in brightness, especially with no obvious attenuation or overexposure between the edge and center areas; low light field uniformity is characterized by "bright center and dark edge" (vignetting), local bright spots or abrupt changes in brightness gradient, which can easily cause visual fatigue, attention shift, or decreased immersion.

[0207] It should be noted that, in this embodiment, the illumination intensity distribution data refers to the illumination intensity sampling set of the display screen of the head-mounted display device within a two-dimensional or quasi-two-dimensional spatial region. Specifically, it is represented by a light intensity matrix or light intensity vector composed of the illumination intensity values ​​corresponding to multiple spatial sampling points, used to characterize the brightness distribution characteristics of the virtual image in each region within the field of view. This data can be collected by a photosensitive sensor array integrated inside the head-mounted display device, near the light-emitting surface of the display module or the area around the user's eyes. The spatial layout of this photosensitive sensor array can be a two-dimensional grid or a ring distribution, used to monitor the spatial illumination intensity distribution of the display screen of the head-mounted display device in real time. This photosensitive sensor array is typically composed of multiple independent photosensitive elements (such as photodiodes or sub-regions of complementary metal-oxide-semiconductor image sensors), capable of simultaneously collecting illumination intensity values ​​at different locations and outputting illumination intensity distribution data (which can be a two-dimensional or quasi-two-dimensional light intensity matrix) composed of the illumination intensity values ​​at different locations, reflecting the brightness distribution of the virtual image in each region within the field of view.

[0208] It should also be noted that, in this embodiment, the light field uniformity information refers to information used to describe the uniformity of light field spatial brightness (i.e., light field uniformity), which can be a light field uniformity score (the higher the score, the better the light field uniformity), a light field uniformity level (the higher the level, the better the light field uniformity), etc.

[0209] In this embodiment, because the optical system (especially the dynamically focused variable focus lens) may introduce non-uniform optical path difference, edge attenuation or local focus deviation during the curvature change process, the displayed image may show uneven light field phenomena such as "bright center and dark edges" or local overexposure / underexposure, which affects the visual realism and comfort.

[0210] Current light field uniformity information refers to the light field uniformity information calculated based on real-time collected light intensity distribution data, which is used to characterize the spatial brightness consistency of the current light field.

[0211] Taking the score of light field uniformity (hereinafter referred to as the light field uniformity index) as an example, the current calculation of light field uniformity information can adopt a variety of statistical methods, such as:

[0212] Standard deviation normalization method: After normalizing the light intensity distribution data, calculate its standard deviation, and then perform nonlinear transformation to obtain the light field uniformity index with a value range of 0 to 1 or 0 to 100. The smaller the standard deviation, the higher the light field uniformity index value and the better the light field uniformity.

[0213] Edge-Center Ratio Method: Calculate the ratio of the average light intensity in the edge region to the average light intensity in the center region of the field of view. After performing a nonlinear transformation, obtain the light field uniformity index with a value range of 0 to 1 or 0 to 100. The closer the ratio is to 1, the higher the light field uniformity index value and the better the light field uniformity.

[0214] Image processing-based uniformity scoring: Treating light intensity distribution as a grayscale image, extracting texture features through methods such as Laplacian operator, gradient magnitude or local contrast analysis, and evaluating uniformity in combination with a preset model.

[0215] In addition to the three methods mentioned above, the current light field uniformity information can also be calculated in other ways. For example, by calculating the absolute deviation between the light intensity of each sampling point and the average light intensity of the entire field in the light intensity distribution data, and normalizing it relative to the average light intensity of the entire field, the relative deviation of the light intensity of each sampling point can be obtained. Then, the sum of the squares of the relative deviations of the light intensity of all sampling points can be averaged to obtain the normalized variance. Then, a nonlinear transformation can be performed to obtain the light field uniformity index with a value range of 0 to 1 or 0 to 100. The smaller the normalized variance, the higher the light field uniformity index value, and the better the light field uniformity.

[0216] The formula for calculating the normalized variance is as follows:

[0217] ;

[0218] in:

[0219] U target Normalized variance;

[0220] N: Number of sampling points;

[0221] I i : Illumination intensity at the i-th sampling point;

[0222] Average light intensity.

[0223] Step S500: Obtain preset target light field uniformity information, and adjust the position of the movable grating based on the target light field uniformity information and the current light field uniformity information.

[0224] In this embodiment, the target light field uniformity information is an ideal light field uniformity standard that is preset by the system or configurable by the user. It represents the level of light field uniformity that the head-mounted display device should achieve in the best display state, and is usually obtained based on optical design simulation or user comfort test calibration.

[0225] It should be noted that, in this embodiment, the movable grating refers to an optical control structure that can be precisely displaced. It is usually composed of a set of periodically arranged light-transmitting and light-blocking strips, integrated into the optical path of the head-mounted display device (such as between the display screen and the lens or between the lens group). It can be controlled by a mechanical drive device, an electromagnetic drive device or a piezoelectric drive device (such as a piezoelectric ceramic actuator) to achieve high-precision multi-degree-of-freedom movement, change its relative spatial position with the light field, thereby selectively attenuating or modulating light in different areas and directions, and realizing dynamic compensation for the overall light field distribution.

[0226] It is understood that the movable grating can also be a grating set on a moving platform, and the moving platform can be controlled by a mechanical drive device, an electromagnetic drive device or a piezoelectric drive device (such as a piezoelectric ceramic actuator) to perform high-precision multi-degree-of-freedom motion, thereby changing the relative spatial position of the grating and the light field.

[0227] This embodiment calculates the light field uniformity deviation between the current light field uniformity information and the target light field uniformity information, and then generates a grating control signal based on this deviation to adjust the position of the movable grating, thereby achieving dynamic optimization of the light field uniformity. For example, when the edges of the image are detected to be too dark, the movable grating is driven to move in a specific direction to reduce occlusion in the edge area or adjust the diffraction effect and improve the edge brightness; if the center of the image is too bright, the light intensity attenuation in the central area is appropriately increased.

[0228] Compared to the first embodiment, which focuses on "focus depth matching" to alleviate the VAC effect, the second embodiment of this application further expands the technical dimensions of optical adjustment and constructs a dual closed-loop adjustment system for "depth-brightness" synergistic optimization. Its technical contributions are reflected in the following three aspects:

[0229] Active compensation and dynamic optimization of light field quality: This embodiment, for the first time, treats light field uniformity as a measurable and adjustable independent optical parameter. Through a closed-loop mechanism of "sensing-evaluation-control," it actively identifies and compensates for secondary optical defects caused by dynamic focusing (such as changes in lens curvature). This mechanism overcomes the limitation of traditional head-mounted displays that only focus on focal length adjustment while neglecting image quality stability, significantly improving visual fidelity and consistency during dynamic focusing.

[0230] By introducing a movable grating as a light field shaping actuator, the dimensions of hardware adjustment are expanded: By integrating this novel optical control element, a movable grating, this embodiment achieves precise intervention in the spatial distribution of the light field. Unlike fixed gratings or static filters, the movable grating possesses dynamic, reversible, and localized control capabilities, enabling targeted correction of light field distortion without sacrificing overall brightness. This demonstrates a deep integration of hardware design and intelligent control, providing a new technical path for optimizing the optical performance of head-mounted displays.

[0231] Constructing a multi-objective collaborative intelligent optical adjustment system: This embodiment unifies the two major objectives of "reducing VAC" and "improving light field uniformity" under the same technical framework, forming a front-to-back linkage adjustment process: the first stage (steps S100~S300 above) ensures that the focus depth matches the needs of the human eye, and the second stage (S400~S500 above) ensures that a high-quality light field distribution is maintained while focusing correctly. This layered and collaborative adjustment strategy enables the head-mounted display device to not only "see clearly" (accurate focus) but also "see comfortably" (uniform brightness and no glare), comprehensively improving the overall quality of the immersive visual experience.

[0232] In summary, the second embodiment of this application is not a simple functional extension of the first embodiment, but rather a significant evolution from "single-point focusing" to "global light field optimization." By introducing light field uniformity sensing and a movable grating control mechanism, this embodiment effectively solves the technical contradiction of "focus imbalance light" in dynamic optical systems, significantly enhancing the imaging stability and visual comfort of head-mounted displays in complex usage scenarios, and constituting a substantial improvement and innovation to existing optical adjustment technologies.

[0233] It is worth mentioning that, in the above-mentioned adjustment of the position of the movable grating, in addition to converting the calculated standard deviation, ratio, and normalized variance into the current light field uniformity information through a specific nonlinear transformation, and then adjusting the position of the movable grating based on the current light field uniformity information and the preset target light field uniformity information, it is also possible to directly generate a grating control signal based on the calculated standard deviation, ratio, or normalized variance, and correspondingly combine it with the preset target standard deviation, target ratio, or target normalized variance, in order to adjust the position of the movable grating.

[0234] In one feasible implementation, the step of adjusting the position of the movable grating based on the target light field uniformity information and the current light field uniformity information in step S500 above may include steps S510~S520:

[0235] Step S510: Calculate the light field uniformity error based on the target light field uniformity information and the current light field uniformity information;

[0236] It should be noted that, in this embodiment, the optical field uniformity error refers to the optical field uniformity deviation between the current optical field uniformity information and the target optical field uniformity information, which is used to quantify the optimization space of the optical field uniform distribution.

[0237] Step S520: Input the light field uniformity error to the preset PID controller to obtain the grating control signal output by the PID controller, and adjust the position of the movable grating through the grating control signal.

[0238] In this embodiment, to achieve high-precision, stable, and low-overshoot position control of the movable grating, a PID (Proportional-Integral-Derivative) controller is introduced as the core algorithm module for closed-loop adjustment of the light field uniformity.

[0239] Specifically, the system uses the light field uniformity error ΔU(t) calculated in step S510 as the input signal of the PID controller, and the controller generates the grating control signal u(t) according to the following control law:

[0240] ;

[0241] in:

[0242] K p Proportional gain reflects the direct impact of the current error on the control output;

[0243] K i Integral gain is used to eliminate steady-state errors and ensure long-term adjustment accuracy.

[0244] K d Differential gain is used to predict the trend of error changes and suppress system overshoot and oscillation.

[0245] The grating control signal u(t) can be expressed as a voltage, pulse width modulation signal or digital command, driving the actuator of the movable grating (such as a micro stepper motor, piezoelectric actuator or electromagnetic coil) to make precise displacement, thereby changing its position and / or angle in the optical path, and realizing dynamic shaping and compensation of the optical field distribution.

[0246] This embodiment, based on the second embodiment, further introduces a closed-loop grating adjustment mechanism based on PID control. Its technical contributions are reflected in the following three aspects:

[0247] Achieving automation and high stability in light field uniformity adjustment: Compared with open-loop control or lookup table method, PID control can dynamically adjust the output according to real-time error, with good robustness and anti-interference ability. It can effectively deal with light field fluctuations caused by changes in ambient light, device aging or user wearing posture deviation, and ensure that the adjustment process converges quickly, with no steady-state error and low oscillation.

[0248] Improving the accuracy and response performance of grating control: The PID controller achieves rapid response through the proportional term, eliminates long-term deviations through the integral term, and suppresses overshoot through the derivative term. The three work together to make the movement of the movable grating smoother and more precise, avoiding the "flickering" or "jittering" phenomenon caused by excessive adjustment or lag, and significantly improving the comfort of the user experience.

[0249] A hierarchical and collaborative multi-objective optical optimization architecture is constructed: This scheme decouples "focal length matching" (first embodiment) and "light field shaping" (second embodiment) into two independent but collaborative control loops: the former uses an AI model to drive a variable focal length lens to achieve millisecond-level dynamic focusing, while the latter uses a PID controller to drive a movable grating to achieve microsecond-to-millisecond-level light field compensation. This hierarchical control strategy of "intelligent feedforward + fine feedback" takes into account the needs of complex intent recognition and high-precision execution control, reflecting the advanced design concept of system-level engineering optimization.

[0250] In summary, this implementation method, by introducing the PID mechanism from classical control theory, elevates the light field uniformity adjustment from "empirical adjustment" to "automatic closed-loop control," significantly enhancing the system's engineering practicality and reliability. This design not only expands the technical dimensions of optical control in head-mounted displays but also provides a replicable technical paradigm for multi-objective collaborative optimization of dynamic optical systems.

[0251] In one feasible implementation, after step S500 described above, the optical adjustment method may further include steps F10 to F20:

[0252] Step F10: Based on the position of the movable grating, update the distortion parameters of the head-mounted display device to obtain the updated distortion parameters;

[0253] Step F20: Inject the updated distortion parameters into the rendering pipeline of the head-mounted display device.

[0254] In this embodiment, when the position of the movable grating changes, the distortion parameters originally applicable to the current optical system will no longer be accurate. This is because the movable grating, as a dynamically adjustable optical element in the optical path, introduces additional optical path differences, diffraction effects, or local light deflection when its position changes, thereby altering the point spread function (PSF) and imaging geometry of the overall optical system. This change not only affects the spatial brightness distribution of the light field but also causes a dynamic shift in the geometric deformation characteristics of the image, manifested as local stretching, compression, or reconstruction of nonlinear distortion fields. If the distortion correction parameters in the rendering pipeline are not updated in time, the image seen by the user will exhibit a "pre-distortion mismatch" phenomenon. That is, the image processed by pre-distortion fails to be restored to a distortion-free state after passing through the updated optical system, instead producing new visual artifacts, such as blurred edges, distorted text, or spatial misalignment, severely affecting visual realism and information readability.

[0255] Therefore, to ensure that the image geometric fidelity remains unaffected while the movable grating performs optical field uniformity compensation, this implementation proposes a dynamic mapping mechanism of "grating state - distortion parameters". Specifically, after the system completes the adjustment of the movable grating's position, it immediately acquires its current position information (such as displacement, rotation angle, and other multi-degree-of-freedom attitude parameters) and uses it as one of the input features. Combined with the curvature state of the current variable focal length lens (i.e., the target radius of curvature), this information is input into a pre-trained distortion parameter prediction model to predict the updated distortion parameters that match the current complete optical configuration.

[0256] It is easy to understand that in this embodiment, the distortion parameter prediction samples used when training the distortion parameter prediction model take the lens curvature of the liquid lens array, the wearer's gaze information (including gaze point coordinates, gaze depth, convergence angle, etc.), and the position of the movable grating as sample features, and the optimal distortion parameters actually measured or simulated under the corresponding conditions as sample labels. Therefore, the position of the movable grating, the target radius of curvature, and the gaze information can be used as model inputs to the distortion parameter prediction model to obtain the target distortion parameters predicted by the distortion parameter prediction model, update the distortion parameters of the head-mounted display device, and then inject the target distortion parameters as the updated distortion parameters into the rendering pipeline of the head-mounted display device.

[0257] It is worth mentioning that while performing step F20, other visual parameters in the rendering pipeline, such as depth-of-field blur radius, dispersion compensation coefficient, and brightness gain mapping table, also need to be adjusted simultaneously to ensure that after the geometric distortion correction is updated, all visual effects are consistent with the overall state of the current optical system, avoiding rendering inconsistencies or perceptual conflicts caused by parameter asynchrony.

[0258] Based on the aforementioned second embodiment's "dynamic compensation for light field uniformity," this implementation further introduces a "grating pose-distortion parameter linkage update" mechanism, constructing a four-level closed-loop optical optimization system of "sensing-adjustment-compensation-recorrection." Its technical contributions are reflected in the following three aspects:

[0259] Solving the problem of complex distortion in multi-degree-of-freedom dynamic optical systems: Traditional head-mounted displays typically only perform distortion calibration on fixed or single-variable optical elements, making it difficult to cope with the complex distortion effects caused by the combined operation of movable gratings and variable-focus lenses. This implementation establishes a dynamic mapping relationship between grating pose and distortion parameters, enabling real-time distortion prediction and compensation for multi-variable coupled systems, significantly improving image fidelity under complex optical architectures.

[0260] Achieving end-to-end synergy between "hardware adjustment" and "software correction": This solution, for the first time, incorporates the state of a movable grating—an active optical control element—into the input variable for distortion correction, breaking down the information barrier between "optical adjustment" and "image preprocessing." This deep integration of hardware and software design enables the system to maintain high-precision geometric consistency while performing light field shaping, demonstrating the system-level integration capability of the "perception-decision-execution-feedback" closed-loop control in next-generation intelligent head-mounted displays.

[0261] Enhancing the overall robustness and user experience consistency of dynamic optics systems: By automatically triggering distortion parameter updates and injection processes after each grating position adjustment, the system can continuously maintain a precise match between the rendered image and the optical imaging result, avoiding visual jumps or distortions caused by grating movement. Combined with a low-latency AI prediction model and an efficient parameter injection mechanism, the entire update process can be completed in sub-milliseconds, achieving a "seamless adjustment, seamless correction" user experience, fundamentally solving the technical bottleneck of "adjusting the light but distorting the image" in dynamic optics systems.

[0262] In summary, this implementation method is not a simple superposition of previous functions, but a profound understanding and systematic response to the relationship between "optical state integrity" and "rendering consistency". It incorporates the physical displacement (including pose changes) of the movable grating into the decision-making loop of image preprocessing, constructing a truly "full-state perception and full-link adaptation" intelligent optical rendering system. This significantly improves the overall imaging performance and visual comfort of head-mounted displays in highly dynamic and multi-target adjustment scenarios, constituting an important improvement and innovation to existing technologies.

[0263] In a further example, the head-mounted display integrates an optical sensing system, whose core components include an event-based visual sensor arranged symmetrically on the left and right, a non-contact ciliary muscle EMG sensor, a near-infrared visual sensor, a photosensitive sensor array, a photonic crystal grating moving platform (i.e., a movable grating), a liquid lens array (i.e., a zoom lens), a pancake lens and a display screen, as well as a time-of-flight camera located on the outside of the head-mounted display facing the wearer's field of vision.

[0264] For dynamic adjustment of optical parameters, the head-mounted display (HMD) can achieve real-time switching of the virtual image focal length through a built-in calibration algorithm. Specifically, when the user needs to switch from focusing on virtual image A (such as a near-field virtual object) to virtual image B (such as a distant virtual scene), the HMD utilizes the rapid response characteristics of the liquid lens array to adjust the curvature and change the focal length. Simultaneously, the photonic crystal grating moving platform optimizes the light field uniformity to match the light intensity requirements under the new focal length. Conversely, when returning from virtual image B to virtual image A, the HMD can also complete the reverse adjustment with the same precision and speed, ensuring the continuity and comfort of the user's visual experience.

[0265] Based on the aforementioned hardware architecture, in the wearing state, the head-mounted display automatically triggers data acquisition: left and right event-based visual sensors acquire dynamic information from the wearer's eyes at high temporal resolution, generating an eye-tracking event stream; left and right non-contact ciliary muscle EMG sensors are closely fitted to the wearer's ciliary muscle surface projection area, acquiring ciliary muscle EMG signals reflecting changes in ciliary muscle tension during lens accommodation in real time; left and right near-infrared visual sensors sample the wearer's pupil diameter at a preset frequency, obtaining pupil diameter sampling data; a time-of-flight camera acquires a scene depth map from the wearer's current perspective. Based on the above data, combined with a pre-trained optical parameter prediction module, the curvature of the left and right liquid lens arrays is adjusted. After curvature adjustment, the left and right photosensor arrays acquire light intensity distribution data, which is processed by a PID controller and outputs a grating control signal to adjust the position and angle of the photonic crystal grating moving platform.

[0266] like Figure 3 As shown, in one example, the head-mounted display is divided into left and right eye regions, each containing cameras (including an event-based visual sensor, a near-infrared visual sensor, and a time-of-flight camera), lenses (i.e., a liquid lens array), VAC adjustment modules, and motors. The left and right VAC adjustment modules apply driving voltage to the left and right lenses to achieve curvature adjustment. The left and right motors control the position and angle of the left and right lenses, thereby changing the curvature of the lenses.

[0267] It should be noted that the above embodiments / implementations are only for understanding this application and do not constitute a limitation on the optical adjustment method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0268] This application provides a head-mounted display device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the optical adjustment method in Embodiment 1 above.

[0269] The head-mounted display devices in the embodiments of this application may include, but are not limited to, head-mounted display devices such as Mixed Reality (MR) devices (e.g., MR glasses or MR helmets), Augmented Reality (AR) devices (e.g., AR glasses or AR helmets), Virtual Reality (VR) devices (e.g., VR glasses or VR helmets), Extended Reality (XR) devices, or some combination thereof.

[0270] like Figure 4 As shown, the head-mounted display device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the head-mounted display device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the head-mounted display to communicate wirelessly or wiredly with other devices to exchange data. While head-mounted display devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0271] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0272] The head-mounted display device provided in this application, employing the optical adjustment method described in the above embodiments, can solve the technical problem of how to more precisely match the focus state adjustment of the optical system to reduce the VAC effect of the head-mounted display device. Compared with the prior art, the beneficial effects of the head-mounted display device provided in this application are the same as those of the optical adjustment method provided in the above embodiments, and other technical features of this head-mounted display device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0273] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0274] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0275] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the optical adjustment method in the above embodiments.

[0276] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0277] The aforementioned computer-readable storage medium may be included in the head-mounted display device; or it may exist independently and not assembled into the head-mounted display device.

[0278] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the head-mounted display device, cause the head-mounted display device to: acquire data on changes in the wearer's eye state; predict target optical parameters, including a target radius of curvature, based on the data on changes in the eye state using a pre-trained optical parameter prediction model; and adjust the curvature of the variable focus lens based on the target radius of curvature.

[0279] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0280] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0281] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0282] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for performing the above-described optical adjustment method. This solves the technical problem of how to more precisely match the focus state adjustment of the optical system to reduce the VAC effect of the head-mounted display device. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the optical adjustment method provided in the above embodiments, and will not be repeated here.

[0283] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the optical adjustment method described above.

[0284] The computer program product provided in this application solves the technical problem of how to more precisely match the focus state adjustment of the optical system to reduce the VAC effect of the head-mounted display device. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the optical adjustment method provided in the above embodiments, and will not be repeated here.

[0285] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method of optical adjustment, characterized by, The optical adjustment method is applied to a head-mounted device including a variable-focus lens, and the method comprises: obtaining eye state change data of a wearer; according to the eye state change data, predicting a target optical parameter by using a pre-trained optical parameter prediction model, wherein the target optical parameter comprises a target curvature radius; based on the target curvature radius, adjusting the curvature of the variable-focus lens; wherein the head-mounted device further comprises a movable grating, and after the step of adjusting the curvature of the variable-focus lens based on the target curvature radius, the method further comprises: obtaining illumination intensity distribution data of a display screen of the head-mounted device, and calculating current light field uniformity information of the head-mounted device according to the illumination intensity distribution data; obtaining a preset target light field uniformity information, and adjusting the position of the movable grating based on the target light field uniformity information and the current light field uniformity information.

2. The method of optical adjustment of claim 1, wherein, The step of predicting a target optical parameter by using a pre-trained optical parameter prediction model according to the eye state change data comprises: inputting the eye state change data into the pre-trained optical parameter prediction model to predict the target optical parameter.

3. The method of optical adjustment of claim 2, wherein, After the step of inputting the eye state change data into the pre-trained optical parameter prediction model to predict the target optical parameter, the method further comprises: inputting the target curvature radius into a pre-trained distortion parameter prediction model to obtain a target distortion parameter predicted by the distortion parameter prediction model; injecting the target distortion parameter into a rendering pipeline of the head-mounted device.

4. The method of optical adjustment of claim 1, wherein, The step of predicting a target optical parameter by using a pre-trained optical parameter prediction model according to the eye state change data comprises: obtaining environmental perception depth data of the wearer; inputting the eye state change data and the environmental perception depth data into the pre-trained optical parameter prediction model to predict the target optical parameter.

5. The method of optical adjustment of claim 4, wherein, After the step of inputting the eye state change data and the environmental perception depth data into the pre-trained optical parameter prediction model to predict the target optical parameter, the method further comprises: obtaining gaze information of the wearer, wherein the gaze information comprises gaze point coordinates, gaze depth, and vergence angle; inputting the target curvature radius and the gaze information into a pre-trained distortion parameter prediction model to obtain a target distortion parameter predicted by the distortion parameter prediction model; injecting the target distortion parameter into a rendering pipeline of the head-mounted device.

6. The method of optical adjustment of claim 1, wherein, After the step of adjusting the position of the movable grating based on the target light field uniformity information and the current light field uniformity information, the method further comprises: updating the distortion parameter of the head-mounted device based on the position of the movable grating to obtain an updated distortion parameter; injecting the updated distortion parameter into a rendering pipeline of the head-mounted device.

7. The method of optical adjustment of claim 1, wherein, The step of adjusting the position of the movable grating based on the target light field uniformity information and the current light field uniformity information comprises: According to the target light field uniformity information and the current light field uniformity information, a light field uniformity error is calculated; The light field uniformity error is input into a preset PID controller to obtain a grating control signal output by the PID controller, and the position of the movable grating is adjusted through the grating control signal.

8. The method of optical adjustment of claim 1, wherein, The target optical parameter includes a target lens focal length, and before the step of adjusting the variable focus lens based on the target radius of curvature, the method further comprises: Obtaining the material refractive index of the variable focus lens, and based on the material refractive index, the target lens focal length and the target radius of curvature are checked by a thin lens focal length formula to obtain a thin lens focal length formula checking result; In the case where the thin lens focal length formula checking result is checked, the step of adjusting the variable focus lens based on the target radius of curvature is executed.

9. The method of optical adjustment of claim 1, wherein, After the step of adjusting the variable focus lens based on the target radius of curvature, the method further comprises: Obtaining the comfort score of the wearer, and calculating a rendering quality indicator of the head-mounted device; The comfort score and the rendering quality indicator are input into a preset first reward function to obtain a first reward value output by the first reward function; Based on the first reward value, the weight parameters of the optical parameter prediction model are updated.

10. The method of optical adjustment of claim 9, wherein, The head-mounted device comprises an automatic scoring module, and the step of obtaining the comfort score of the wearer comprises: Obtaining ciliary muscle EMG data of the wearer, and calculating ciliary muscle tension of the wearer according to the ciliary muscle EMG data; Obtaining pupil diameter sampling data of the wearer, and calculating a pupil diameter change rate of the wearer according to the pupil diameter sampling data; The ciliary muscle tension and the pupil diameter change rate are input into the automatic scoring module to obtain a first comfort score output by the automatic scoring module; Based on the first comfort score, the comfort score of the wearer is determined.

11. The method of optical adjustment of claim 10, wherein, The head-mounted device further comprises an active scoring module, and the step of determining the comfort score of the wearer based on the first comfort score comprises: Obtaining a second comfort score submitted by the wearer through the active scoring module; The first comfort score and the second comfort score are weighted and fused to obtain the comfort score of the wearer.

12. The method of optical adjustment of claim 3 or 5, wherein, After the step of injecting the target distortion parameter into the rendering pipeline of the head-mounted device, the method further comprises: Obtaining the comfort score of the wearer, and calculating a rendering quality indicator of the head-mounted device; The comfort score and the rendering quality indicator are input into a preset second reward function to obtain a second reward value output by the second reward function; Based on the second reward value, the weight parameters of the distortion parameter prediction model are updated.

13. A head-mounted device, comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the optical adjustment method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the optical adjustment method according to any one of claims 1 to 12.

Images