Optical adjustment method, head-mounted display device and storage medium

Abstract

The invention discloses an optical adjustment method, head-mounted display equipment and a storage medium, and relates to the technical field of head-mounted display equipment, the optical adjustment method is applied to the head-mounted display equipment, and the head-mounted display equipment comprises a biological feature acquisition module and a liquid lens array; the method comprises the following steps: acquiring target biological characteristic data of a wearer through a biological characteristic acquisition module; the target biological characteristic data are input into a preset optical parameter prediction neural network model, target optical parameters are obtained through prediction, and the optical parameter prediction neural network model is obtained by taking the biological characteristic data as a sample and taking the optical parameters as label training; and adjusting the control voltage of the liquid lens array based on the target optical parameter so as to adjust the target curvature of the lens in the liquid lens array. The head-mounted display device can adapt to biological characteristic data of different users, the focal length of the head-mounted display device is automatically adjusted, and the VAC effect of the head-mounted display device is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of head-mounted display devices, and in particular to an optical adjustment method, a head-mounted display device, and a storage medium. Background Art

[0002] With the rapid development of head-mounted display (HMD) technology, the optimization of HMD performance and user experience has become the focus of industry research.

[0003] Traditional head-mounted displays (HMDs) rely primarily on a fixed focal length design or manual user adjustment for visual adjustment. This static adjustment mode cannot flexibly adapt to the individual biometric differences of different users, such as key parameters like pupil distance and corneal curvature. Due to the significant differences in biometric characteristics between individuals, fixed focal length or manual adjustment often makes it difficult to achieve precise matching, which in turn leads to a significant increase in visual convergence-accommodation conflict (VAC effect). The VAC (Vergence-Accommodation Conflict) effect manifests itself as the user's eyes cannot naturally converge on the same focal point when trying to focus on objects in a virtual scene, causing visual fatigue, dizziness, and other discomfort.

[0004] Therefore, how to reduce the VAC effect of head-mounted display devices has become an urgent problem to be solved by those skilled in the art.

[0005] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0006] The main purpose of this application is to provide an optical adjustment method, a head-mounted display device, and a storage medium, aiming to solve the technical problem of how to reduce the VAC effect of the head-mounted display device.

[0007] To achieve the above objectives, the present application proposes an optical adjustment method, which is applied to a head-mounted display device, wherein the head-mounted display device includes: a biometric feature acquisition module and a liquid lens array; the method includes: Collecting target biometric data of the wearer through the biometric collection module; Inputting the target biometric data into a preset optical parameter prediction neural network model to predict target optical parameters, wherein the optical parameter prediction neural network model is trained using the biometric data as a sample and the optical parameters as labels; A control voltage of the liquid lens array is adjusted based on the target optical parameter to adjust the curvature of the lenses in the liquid lens array.

[0008] In one embodiment, the target optical parameters include: focal length compensation value, lens curvature and dispersion compensation coefficient; The step of adjusting the control voltage of the liquid lens array based on the target optical parameters comprises: Calculating a curvature correction term according to the focal length compensation value and the dispersion compensation coefficient; Adding the lens curvature and the curvature correction term to obtain a target curvature of the liquid lens array; Calculating a curvature difference between an initial curvature of the liquid lens array and a target curvature; Calculating a target voltage based on the curvature difference and a preset curvature-voltage coefficient; The control voltage is adjusted according to the target voltage.

[0009] In one embodiment, before the step of inputting the target biometric data into a preset optical parameter prediction neural network model, the method further comprises: If the target biometric data does not conform to a preset normal range, recollect the target biometric data; The target biometric data is input into a preset convolutional network to obtain vectorized target biometric data.

[0010] In one embodiment, the head-mounted display device further includes: a score acquisition module; After the step of adjusting the control voltage of the liquid lens array based on the target optical parameters, the method further includes: Obtaining a comfort score through the score acquisition module; The optical parameter prediction neural network model is updated based on the comfort score, and the step of collecting the wearer's target biometric data through the biometric collection module is returned to be executed.

[0011] In one embodiment, the score acquisition module includes: a physiological signal acquisition unit; The step of obtaining the comfort score through the score obtaining module includes: collecting target physiological signals of the wearer through the physiological signal collecting unit; The comfort score corresponding to the target physiological signal is determined according to a preset mapping relationship.

[0012] In one embodiment, the target physiological signals include: blink frequency and skin conductance; The step of determining the comfort score corresponding to the target physiological signal according to the preset mapping relationship includes: Determining a first comfort score corresponding to the blinking frequency and a second comfort score corresponding to the skin conductance according to a preset mapping relationship, wherein the blinking frequency is negatively correlated with the first comfort score, and the skin conductance is negatively correlated with the second comfort score; The comfort score is obtained by performing weighted calculation on the first comfort score and the second comfort score according to a preset weight.

[0013] In one embodiment, the optical parameter prediction neural network model is a GAN model, and the step of updating the optical parameter prediction neural network model based on the comfort score includes: Determining a loss function of the GAN model according to the comfort score; The weights of the generator in the GAN model are updated based on a preset gradient descent algorithm and the loss function.

[0014] In one embodiment, the biometric feature acquisition module includes: a camera unit and an eye movement detection unit; the target biometric feature data includes: pupil distance, corneal curvature, gaze point coordinates and eye movement speed; The step of collecting target biometric data of the wearer through the biometric collection module includes: Acquiring an eye image of the wearer through the camera unit; acquiring the pupil distance and corneal curvature of the wearer based on the eyeball image; The wearer's gaze point coordinates and eye movement speed are obtained through the eye movement detection unit.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a head-mounted display device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the optical adjustment method described above.

[0016] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the optical adjustment method described above are implemented.

[0017] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the optical adjustment method described above are implemented.

[0018] The present application provides an optical adjustment method, which collects target biometric data of the wearer through a biometric collection module carried by a head-mounted display device, inputs the target biometric data into an optical parameter prediction neural network model that has been pre-trained using the biometric data as samples and optical parameters as labels, obtains the target optical parameters output by the optical parameter prediction neural network model, and then adjusts the control voltage of the liquid lens array of the head-mounted display device based on the target optical parameters to adjust the target curvature of the lens in the head-mounted display device, thereby reducing the VAC effect of the head-mounted display device.

[0019] In summary, the present application uses a pre-trained optical parameter prediction neural network model to predict optical parameters adapted to the user's biometric data based on the collected target biometric data of the wearer, and then adjusts the control voltage of the liquid lens array of the head-mounted display device based on the optical parameters to adjust the target curvature of the lens in the head-mounted display device. Compared with a fixed focal length or manual adjustment of the focal length by the user, the present application enables the head-mounted display device to adapt to the biometric data of different users, automatically adjust the focal length of the head-mounted display device, and reduce the VAC effect of the head-mounted display device. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A schematic diagram of a flow chart of the first embodiment of the optical adjustment method of the present application; Figure 2 This is a schematic diagram of the structure of a head-mounted display device involved in an embodiment of the optical adjustment method of this application; Figure 3 A schematic diagram of the feedback process provided in Example 2 of the optical adjustment method of this application; Figure 4 A schematic diagram of a flow chart of Embodiment 2 of the optical adjustment method of this application; Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the optical adjustment method in the embodiment of the present application.

[0023] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0025] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0026] The main solution of the embodiment of the present application is: collecting the wearer's target biometric data through the biometric collection module; inputting the target biometric data into a preset optical parameter prediction neural network model to predict target optical parameters, wherein the optical parameter prediction neural network model is trained using the biometric data as samples and the optical parameters as labels; adjusting the control voltage of the liquid lens array based on the target optical parameters to adjust the target curvature of the lenses in the liquid lens array.

[0027] With the rapid development of head-mounted display (HMD) technology, the optimization of HMD performance and user experience has become the focus of industry research.

[0028] Traditional head-mounted displays (HMDs) rely primarily on a fixed focal length design or manual user adjustment for visual adjustment. This static adjustment mode cannot flexibly adapt to the individual biometric differences of different users, such as key parameters like pupil distance and corneal curvature. Due to the significant differences in biometric characteristics between individuals, fixed focal length or manual adjustment often makes it difficult to achieve precise matching, which in turn leads to a significant increase in visual convergence-accommodation conflict (VAC effect). The VAC (Vergence-Accommodation Conflict) effect manifests itself as the user's eyes cannot naturally converge on the same focal point when trying to focus on objects in a virtual scene, causing visual fatigue, dizziness, and other discomfort.

[0029] Therefore, how to reduce the VAC effect of head-mounted display devices has become an urgent problem to be solved by those skilled in the art.

[0030] In response to the above problems, the present application provides an optical adjustment method, which collects the wearer's target biometric data through a biometric collection module carried by a head-mounted display device, inputs the target biometric data into an optical parameter prediction neural network model that has been pre-trained using the biometric data as samples and optical parameters as labels. After obtaining the target optical parameters output by the optical parameter prediction neural network model, the control voltage of the liquid lens array of the head-mounted display device is adjusted based on the target optical parameters to adjust the target curvature of the lens in the head-mounted display device, thereby reducing the VAC effect of the head-mounted display device.

[0031] In summary, the present application uses a pre-trained optical parameter prediction neural network model to predict optical parameters adapted to the user's biometric data based on the collected target biometric data of the wearer, and then adjusts the control voltage of the liquid lens array of the head-mounted display device based on the optical parameters to adjust the target curvature of the lens in the head-mounted display device. Compared with a fixed focal length or manual adjustment of the focal length by the user, the present application enables the head-mounted display device to adapt to the biometric data of different users, automatically adjust the focal length of the head-mounted display device, and reduce the VAC effect of the head-mounted display device.

[0032] The head-mounted display device in the embodiments of this application may include, but is not limited to, mixed reality (MR) devices (e.g., MR glasses or MR helmets), augmented reality (AR) devices (e.g., AR glasses or AR helmets), virtual reality (VR) devices (e.g., VR glasses or VR helmets), extended reality (XR) devices, or some combination thereof, and the like. In this embodiment, for ease of description, the following description uses the head-mounted display device as the execution subject.

[0033] Based on this, the embodiment of the present application provides an optical adjustment method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the optical adjustment method of the present application.

[0034] In this embodiment, the optical adjustment method is applied to a head-mounted display device, which includes: a biometric feature acquisition module and a liquid lens array; the optical adjustment method includes steps S10 to S30: Step S10, collecting target biometric data of the wearer through the biometric collection module; It should be noted that, in this embodiment, the technologies used by the head-mounted display device (i.e., head-mounted display device) mainly include AR (Augmented Reality) technology, VR (Virtual Reality) technology, and MR (Mixed Reality) technology. Based on these technologies, when a user wears a head-mounted display device, they can display virtual content in a real scene through various sensors, camera devices, or optical components in the device.

[0035] It should be noted that, in this embodiment, the biometric collection module may include multiple hardware modules for collecting different types of biometric data, has biometric recognition capabilities, and is used to collect multiple key biometric data of the wearer.

[0036] In this embodiment, after detecting that the user is wearing the head-mounted display device correctly, the built-in biometric data collection module of the head-mounted display device automatically starts the biometric data collection function to collect the wearer's biometric data related to the VAC effect, including but not limited to inter-pupillary distance (IPD), corneal curvature, iris texture, etc.

[0037] In a feasible implementation, the setting position of the liquid lens array may also be adjusted based on the collected IPD.

[0038] In this embodiment, the liquid lens array can be set in a set of mechanical structures whose positions can be adjusted separately. After collecting the wearer's IPD, the setting position of the liquid lens array is adjusted according to the IPD so that the visual center of the liquid lens array matches the IPD, further adapting to the user's physiological data.

[0039] Furthermore, in a feasible embodiment, the biometric feature collection module includes: a camera unit and an eye movement detection unit; the target biometric feature data includes: pupil distance, corneal curvature, gaze point coordinates and eye movement speed; the above-mentioned step S10 may include steps S11 to S15: Step S11, obtaining an eye image of the wearer through a camera unit; It should be noted that, in this embodiment, the biometric feature acquisition module may include a camera unit and an eye movement detection unit. The camera unit may be used to capture the wearer's eye image, and the eye movement detection unit may be used to track the wearer's eye movement data.

[0040] In this embodiment, when the wearer correctly puts on the head-mounted display device, the camera unit starts working. The camera unit usually uses a high-resolution camera with automatic focus and light compensation functions to ensure that clear and accurate eye images can be obtained under different lighting conditions. During the acquisition process, the camera unit will capture the wearer's eye area in real time and obtain a series of continuous eye image frames. These image frames contain eye appearance information, such as pupil size, position, corneal morphology and other characteristics.

[0041] Step S12, obtaining the wearer's pupillary distance and corneal curvature based on the eyeball image; In this embodiment, after capturing the image of the eyeball, the image processing algorithm built into the head-mounted display device analyzes and processes the eyeball image captured by the camera unit to extract biometric data such as pupil distance and corneal curvature.

[0042] As an example, specifically, the algorithm first preprocesses the eye image, including image enhancement, edge detection and other operations to highlight the outline of the pupil. Then, it uses image recognition technology to determine the center positions of the two pupils and calculate the distance between them, which is the pupil distance.

[0043] Then, the corneal morphology information in the eyeball image is used, combined with the preset corneal model and mathematical algorithm, to calculate the corneal curvature radius. Corneal curvature is an important indicator to measure the degree of corneal curvature and is crucial for the optical design of head-mounted display devices.

[0044] Step S13, obtaining the wearer's gaze point coordinates and eye movement speed through the eye movement detection unit; In this embodiment, the eye movement detection unit calculates the coordinates of the wearer's current gaze point based on the eyeball's movement trajectory and pupil position. The gaze point coordinates reflect the wearer's visual focus. In addition to the gaze point coordinates, the eye movement detection unit can also measure the speed of the eyeball's movement, i.e., eye velocity. Changes in eye velocity can reflect information such as the wearer's level of attention and visual search efficiency.

[0045] In addition, the data input into the model can also include the user's historical comfort score, including: Step S14, obtaining multiple historical reference scores according to a preset window; In this embodiment, in order to more comprehensively evaluate the wearer's comfort perception, the head-mounted display device will also consider the wearer's historical usage data and obtain multiple historical reference scores from the historical data according to a preset time window (such as the last 5 scores, the last month, etc.). These historical reference scores can be the comfort scores obtained by the wearer through the score acquisition module when using the head-mounted display device in the past, or they can be scores automatically calculated by the device based on other indicators (such as usage time, number of abnormal exits, etc.).

[0046] Step S15: Calculate the average of each historical reference score to obtain a historical score.

[0047] In this embodiment, after obtaining multiple historical reference scores, the head-mounted display device will average these scores to obtain a comprehensive historical score. The historical score reflects the wearer's overall comfort feeling of the head-mounted display device over the past period of time. This score can serve as an important reference for current comfort evaluation and help the device better understand the wearer's needs and preferences.

[0048] For further examples, see Figure 2The head-mounted display (HMD) integrates an optical perception system. Its core components include symmetrically arranged left and right infrared camera modules and left and right iris recognition camera arrays. These cameras utilize high-sensitivity sensors with low noise and high dynamic range, enabling them to accurately capture subtle features of the human eye area. The zoom control module utilizes liquid lens array technology. This array consists of multiple miniature liquid lens units, each of which can independently adjust its focal length through electric field drive, enabling continuous zoom with millisecond-level response. Simultaneously, the electrochromic layer precisely controls light transmittance by applying different voltages, dynamically optimizing the optical system's luminous flux and contrast. Through the coordinated action of the zoom module, left and right beam-splitting prisms, and a high-precision optical lens assembly, the virtual image is precisely projected onto the display screen. Distortion is corrected through a free-form optical system, allowing the user to observe a clear, distortion-free virtual image through the eyepiece.

[0049] To dynamically adjust optical parameters, the headset uses a built-in calibration algorithm to achieve real-time switching of virtual image focal length. Specifically, when the user switches from focusing on virtual image A (e.g., a nearby virtual object) to virtual image B (e.g., a distant virtual scene), the headset adjusts the focal length using the rapid response of the liquid lens array. Simultaneously, the electrochromic layer optimizes the transmittance to match the required light intensity at the new focal length. Conversely, when switching from virtual image B back to virtual image A, the headset can also complete the reverse adjustment with the same precision and speed, ensuring a consistent and comfortable visual experience.

[0050] Based on the above hardware architecture, when the head-mounted display device collects biometric data, the left and right infrared cameras work synchronously at a 60Hz sampling rate, and use the binocular stereo vision algorithm to calculate the three-dimensional coordinates of the wearer's pupil center in real time, thereby obtaining an accurate IPD value. This data is stored frame by frame in the headset's built-in flash memory in CSV format (including fields such as timestamp, left / right pupil X / Y / Z coordinates, and IPD value). The iris camera captures dynamic images of the wearer's eyeballs at a high frame rate (≥30fps). The headset uses a deep learning algorithm to identify micro-deformation features on the iris surface (such as vascular distribution and wrinkle morphology), and combines it with a geometric optical model to invert the corneal curvature radius. The collected corneal curvature images are stored in JPEG format (640×480 pixels, 8-bit grayscale). The eye tracker module captures the rotation angle and micro-movements of the wearer's eyeballs at a sampling rate of 120Hz, calculates the coordinates of the gaze point, and uses optical flow to analyze eye movement velocity and acceleration. The detection data is stored in JSON format (including fields such as timestamp, gaze point X / Y coordinates, and eye movement velocity vector), supporting real-time gaze point heat map generation and eye movement trajectory playback.

[0051] Step S20: Input the target biometric data into a preset optical parameter prediction neural network model to predict the target optical parameters, wherein the optical parameter prediction neural network model is trained using the biometric data as samples and the optical parameters as labels; It should be noted that in this embodiment, the head-mounted display device pre-integrates an optical parameter prediction neural network model, which is derived through extensive training, using a neural network model based on a GAN (Generative Adversarial Network) model, an LSTM (Long Short Term Memory) model, a CNN (Convolutional Neural Network) model, or an RNN (Recurrent Neural Network) model. During the model training phase, this optical parameter prediction neural network model utilizes a large amount of biometric data from people of different ages and genders, along with a sample dataset of optical parameters that provide the best wearing experience and visual comfort corresponding to the current biometric signals. Through a deep learning algorithm, the optical parameter prediction neural network model is able to learn the complex mapping relationship between biometric data and optical parameters.

[0052] In this embodiment, the head-mounted display device inputs the collected target biometric data as input data into a preset optical parameter prediction neural network model. The optical parameter prediction neural network model quickly processes and analyzes the input target biometric data, and uses its internally learned mapping relationship to predict and output target optical parameters that match the wearer's biometric data. These target optical parameters directly determine the VAC effect of the head-mounted display device.

[0053] The optical parameter prediction neural network model can be a GAN (Generative Adversarial Network) model, an LSTM (Long Short Term Memory) model, a CNN (Convolutional Neural Network) model, or an RNN (Recurrent Neural Network) model, and this embodiment does not specifically limit this.

[0054] As an example, the optical parameter prediction neural network model can be a GAN (Generative Adversarial Network) model, which includes a generator and a discriminator. The generator includes an input layer (10-dimensional biological feature vector), a fully connected layer (512 nodes, Leaky ReLU), a fully connected layer (256 nodes), and an output layer (3-dimensional optical parameters: focal length compensation value f, lens curvature r, and dispersion coefficient δ).

[0055] The discriminator includes an input layer (3D optical parameters), a convolutional layer (64 filters, stride 2), a fully connected layer (128 nodes), and a binary classification output (true / generated probability). Among the multiple inference results output by the output generator, the set of data with the highest true / generated probability is selected as the final output result.

[0056] Furthermore, in a feasible implementation manner, before the above step S20, the method may further include steps A10 to A20: Step A10: if the target biometric data does not conform to the preset normal range, re-collect the target biometric data; In this embodiment, after collecting the wearer's target biometric data, the head-mounted display device performs a preliminary validity check on this data. The preset normal range is determined based on a large amount of sample data and actual application requirements, covering the range of biometric data under normal conditions for different user groups. For example, a reasonable interval is set for pupillary distance, and a corresponding normal value range is also set for corneal curvature.

[0057] When the headset determines that the collected target biometric data is not within the preset normal range, there may be a variety of reasons, such as sensor failure, interference from the collection environment, and improper wearing of the wearer. At this time, the headset will trigger the re-collection mechanism, informing the wearer through interface prompts or sound prompts to re-collect the biometric data. During the re-collection process, the biometric collection module will start again and obtain new target biometric data according to the same collection process and standards. The headset will re-verify the legitimacy of the re-collected data until the collected data is within the preset normal range, ensuring the accuracy and reliability of the data used in subsequent steps.

[0058] Taking the interpupillary distance (IPD) as an example, the normal range preset for the head-mounted display device can not only include the statistical mean ±2σ interval, but also introduce quantile corrections based on the user's age group (such as allowing ±3σ fluctuations for child users), dynamic compensation for the device wearing angle (±1mm tilt tolerance), and calibration coefficients for real-time ambient light effects (such as the IPD value may shrink by 0.5-1mm in a strong light environment).

[0059] When the head-mounted display device detects through the verification algorithm that the collected data exceeds the preset range, it will trigger a multi-dimensional abnormality diagnosis mechanism. First, the device analyzes the sensor's own status parameters (such as infrared light source intensity, camera exposure time, temperature drift, etc.), and compares them with historical benchmark values ​​to determine whether there is a hardware failure; secondly, it combines environmental sensor data (such as light intensity, humidity, and electromagnetic interference intensity) to evaluate external interference factors; finally, by analyzing the wearer's head posture (accelerometer / gyroscope data), device wearing pressure distribution (pressure sensor array data), and user interaction behavior (such as blink frequency, nystagmus pattern), it determines whether there is improper wearing (such as device slipping, light leakage, pressure on the eye socket) or user physiological abnormalities (such as fatigue, drug effects).

[0060] If an anomaly is determined to be retryable (such as a brief environmental disturbance or a minor misalignment), the headset will immediately initiate a recollection mechanism. The device will use multimodal interaction methods, including visual cues (such as pupil position calibration animations and device wearing angle diagrams), auditory cues (multilingual voice commands, ambient sound compensation), and haptic feedback (mild vibration prompts), to guide the wearer to adjust the device position, clean the collection area, or re-enter the collection state. During the recollection process, the biometric collection module (including the infrared camera array, iris scanner, eye tracker, etc.) will operate in a higher-precision mode, enable self-calibration algorithms (such as autofocus, white balance adjustment, and distortion correction), and increase data redundancy (such as taking the median after collecting three sets of data).

[0061] In step A20, the target biometric data is input into a preset convolutional network to obtain vectorized target biometric data.

[0062] In this embodiment, after the target biometric data passes the legality verification, it needs to be vectorized to facilitate subsequent processing and analysis in the optical parameter prediction neural network model. The preset convolutional network is a deep learning model designed specifically for this purpose.

[0063] Specifically, after collecting the wearer's biometric data, in order to facilitate the analysis and calculation of the subsequent model, the biometric data needs to be preprocessed. First, the abnormal values ​​in the biometric data are removed. For example, when the IPD does not meet the range of 54 to 74 mm, it is considered abnormal data and needs to be re-measured. After all the abnormal data are removed, the parameters such as IPD and corneal curvature are normalized and mapped to the interval of [0, 1]. Then, the multimodal data is synchronized through the timestamp of the collected data, and each data including the user's historical comfort score is input into the convolutional network for feature extraction, and finally a 10-dimensional feature vector is output. , where X IPD : IPD data; X cornea : corneal curvature data; X gaze : Gaze point coordinates, 10-dimensional feature vector includes: 1) interpupillary distance (IPD, 1 dimension); 2) corneal curvature (3D, extracted through iris image features); 3) gaze point coordinates (2D, x / y axis normalized values); 4) eye movement velocity (2-dimensional, horizontal / vertical components); 5) User historical comfort rating (2-dimensional, sliding average of the last 5 ratings).

[0064] Step S30 , adjusting the control voltage of the liquid lens array based on the target optical parameters to adjust the target curvature of the lenses in the liquid lens array.

[0065] In this embodiment, after the head-mounted display device receives the target optical parameters output by the optical parameter prediction neural network model, it immediately starts the control voltage adjustment program of the liquid lens array. The liquid lens array is a key component in the head-mounted display device for achieving optical zoom and aberration correction. The accuracy of its operating parameters directly affects the VAC effect of the head-mounted display device.

[0066] In this embodiment, the head-mounted display device adjusts the control voltage of the liquid lens array in real time through electronic control means based on the received target optical parameters. Through the electrodeformable material, the change in control voltage can cause the lens material to undergo a specific deformation to adjust the curvature of the lens.

[0067] Furthermore, in a feasible implementation, the target optical parameters include: focal length compensation value, lens curvature and dispersion compensation coefficient; the above step S30 may further include steps S31 to S35: Step S31, calculating a curvature correction term according to the focal length compensation value and the dispersion compensation coefficient; It should be noted that, in this embodiment, the focal length compensation value is used to compensate for the deviation between the actual focal length and the ideal focal length caused by differences in the wearer's biometric characteristics (such as pupillary distance and corneal curvature), so as to ensure clear and accurate imaging.

[0068] Lens curvature is used to describe the degree of curvature of the lens surface. Different curvatures will affect the refraction and focusing effects of light.

[0069] The dispersion compensation coefficient is used to correct the dispersion phenomenon caused by the different refractive indices of light of different colors, reduce chromatic aberration, and improve the color reproduction and clarity of the image.

[0070] In this embodiment, the curvature correction term, ie, the curvature that needs to be compensated, can be calculated according to the focal length compensation value and the dispersion compensation coefficient.

[0071] Step S32, adding the lens curvature and the curvature correction term to obtain the target curvature of the liquid lens array; In this embodiment, after the curvature correction term is calculated, the lens curvature and the curvature correction term are added together to obtain the target curvature to which the liquid lens array needs to be adjusted.

[0072] Step S33, calculating the curvature difference between the initial curvature and the target curvature of the liquid lens array; In this embodiment, in order to calculate the control voltage that needs to be adjusted, it is also necessary to calculate the curvature difference between the initial curvature and the target curvature of the lens in the liquid lens array.

[0073] Step S34, calculating the target voltage based on the curvature difference and a preset curvature-voltage coefficient; In this embodiment, after calculating the curvature difference, Magic calculates the target voltage based on a preset curvature-voltage coefficient.

[0074] Step S35: adjusting the control voltage according to the target voltage.

[0075] In this embodiment, after the target voltage is obtained, the control voltage is adjusted to the target voltage to adjust the lens curvature of the liquid lens array.

[0076] Specifically, in one example, the radius of curvature R of the liquid lens is strictly related to the focal length f by the thin lens formula: ; in: n: refractive index of liquid lens material (e.g. PDMS, n=1.43); R fix : The radius of the fixed curvature surface (determined by the mechanical structure of the lens, such as R fix =500 μm); R: dynamically adjusted curvature radius; The relationship between the curvature radius R of the liquid lens and the driving voltage V is: ; in: R: Target curvature radius (μm), calculated from the curvature parameter in the generator output Y.

[0077] V: driving voltage (V), the target value to be solved.

[0078] k: Lens material characteristic constant (e.g., k = 0.12 μm / V2 for dielectric elastomer).

[0079] , :The vacuum dielectric constant (8.85e-12 F / m) and the relative dielectric constant of the material (such as PDMS =2.8).

[0080] d: Electrode spacing (μm, default 50μm).

[0081] R0: Zero voltage curvature radius (determined by the initial shape of the lens, usually 200μm).

[0082] By collecting the wearer's target biometric data and using an optical parameter prediction neural network model to predict optical parameters tailored to the user's biometric data, the head-mounted display (HMD) achieves personalized optical adaptation, ensuring an optimal wearing experience for every user, regardless of their biometric data. Conventional HMDs, due to their fixed focal length or manual adjustment methods, often struggle to achieve precise matching, significantly increasing the visual vergence-accommodation conflict (VAC effect). The optical adjustment method of this application automatically adjusts the operating parameters of the liquid lens array to adapt the HMD's optical parameters to the user's biometric data, effectively reducing the VAC effect and alleviating discomfort such as visual fatigue and dizziness. Furthermore, because the HMD automatically adapts to different users' biometric data and adjusts operating parameters in real time to reduce the VAC effect, users no longer need to perform complex manual adjustments during use. This not only improves ease of use but also significantly enhances the overall user experience.

[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 3 , Figure 3 This is a flow chart of the second embodiment of the optical adjustment method of the present application.

[0084] In this embodiment, the head-mounted display device further includes: a score acquisition module; after the above-mentioned step S30, the optical adjustment method may further include steps S40 to S50: Step S40, obtaining a comfort score through a score obtaining module; In this embodiment, after the head-mounted display device adjusts the operating parameters of the liquid lens array based on the target optical parameters, the device's built-in score acquisition module is used to monitor and evaluate the wearer's comfort perception in real time. Specifically, the score acquisition module can obtain the comfort score in a variety of ways, such as: Physiological indicator monitoring: Use built-in sensors to monitor the wearer's heart rate, skin electrical response and other physiological indicators. These indicators can reflect the wearer's tension and comfort level.

[0085] Visual feedback collection: Through the built-in camera or eye tracking technology of the head-mounted display device, the wearer's visual behaviors such as blinking frequency and gaze movement are observed to infer their visual comfort.

[0086] Active user feedback: Provide a comfort rating option on the user interface of the headset, allowing wearers to rate their comfort based on their subjective feelings.

[0087] The scoring module integrates the above information and calculates the current comfort score through a preset algorithm. The score reflects the wearer's satisfaction and comfort with the current optical parameters of the head-mounted display device.

[0088] Furthermore, in a feasible implementation, the score acquisition module includes: a physiological signal acquisition unit; the above-mentioned step S40 may include steps S41-S42: Step S41, collecting target physiological signals of the wearer through a physiological signal collection unit; In this embodiment, when performing feedback detection, the physiological signal acquisition unit acquires the wearer's target physiological signal.

[0089] Step S42: determining the comfort score corresponding to the target physiological signal according to a preset mapping relationship.

[0090] In this embodiment, a mapping relationship between physiological signals and comfort scores is pre-stored in the head-mounted display device. According to this mapping relationship, the comfort score corresponding to the target physiological signal can be determined.

[0091] Specifically, when evaluating the comfort of head-mounted display (HMD) users, the entire evaluation process integrates interactive technology, biosignal detection technology, and data analysis algorithms. Specifically, in the process of obtaining a comfort score, the HMD first displays an interactive interface in the form of a graphical user interface (GUI) through its built-in display module, guiding the user to complete a series of preset operational tasks or evaluation processes. At the same time, to enhance the diversity and convenience of interaction, the device is also equipped with a speaker system that can output voice prompts to guide users on how to submit their subjective ratings of their current user experience through voice commands, touch clicks, or other preset interaction methods (such as gesture recognition, eye tracking triggers, etc.).

[0092] While the user is interacting with the device, the head-mounted display (HMD) collects the user's physiological data in a non-invasive manner. On the one hand, the device measures changes in the conductivity of the skin around the user's eyes through bioelectric detection sensors integrated around the eyes or frames. Skin conductance is a sensitive indicator of sympathetic nervous system activity, and its changes are closely related to the individual's emotional state, cognitive load, and physiological stress response. When the user experiences discomfort such as visual fatigue, motion sickness, or regulatory conflict while using the HMD, the excitability of the sympathetic nervous system will be significantly enhanced, prompting increased sweat gland secretion, which in turn leads to an increase in skin conductivity. Conversely, in a comfortable usage state, the activity of the parasympathetic nervous system dominates, and skin conductivity tends to stabilize or decrease. Therefore, by continuously monitoring changes in skin conductivity, the user's comfort level can be objectively reflected.

[0093] On the other hand, head-mounted displays also use built-in cameras or dedicated eye-tracking sensors to monitor the user's blinking behavior in real time. Blinking frequency and eye closure duration are another important indicator reflecting the user's physiological state and cognitive load. Their changes can also reveal the user's comfort level. Specifically, when the user feels uncomfortable or tired, the blinking frequency tends to increase and the eye closure duration may also be prolonged. These are the body's natural reactions to try to relieve discomfort by reducing visual stimulation.

[0094] Based on the user's bio-signals obtained from skin conductivity and blinking behavior, the headset uses an algorithmic model to conduct a comprehensive analysis, thereby objectively and accurately evaluating the user's comfort level. To further improve the accuracy and reliability of the assessment, the device also collects scoring data from multiple data sources, at multiple different time points, or in different usage scenarios, and integrates and processes this data using a weighted average method to ultimately produce a comprehensive and objective comfort score.

[0095] Further, in a feasible implementation, please refer to Figure 4, the target physiological signals include: blink frequency and skin conductance; the above step S42 may include steps S421~S422: Step S421: determining a first comfort score corresponding to the blinking frequency and a second comfort score corresponding to the skin conductance according to a preset mapping relationship, wherein the blinking frequency is negatively correlated with the first comfort score, and the skin conductance is negatively correlated with the second comfort score; It should be noted that under normal circumstances, when a person is relaxed and comfortable, their blinking frequency is relatively stable and natural. However, when wearing a head-mounted display (HMD) device and experiencing discomfort, such as visual fatigue or dry eyes, the wearer will often unconsciously increase their blinking frequency in an attempt to alleviate the discomfort. Therefore, a higher blinking frequency indicates a lower wearer's comfort level, and the lower the first comfort score. Similarly, skin conductance reflects the electrical properties of the skin. When the human body is in a state of tension, anxiety, or discomfort, the sweat glands in the skin increase, leading to an increase in skin conductance. If discomfort occurs while wearing a HMD device, the wearer's mood may be affected, which in turn causes changes in skin conductance. Therefore, a higher skin conductance indicates a lower wearer's comfort level, and the lower the second comfort score.

[0096] In this embodiment, through the collection and analysis of extensive experimental data, a mapping relationship between blink frequency and a first comfort score can be established. For example, different blink frequency ranges are set to correspond to different first comfort scores. When the blink frequency is in a lower range, the first comfort score is higher; as the blink frequency increases, the first comfort score gradually decreases. Also based on the experimental data, a mapping relationship between skin conductance and a second comfort score is established. Different skin conductance thresholds or ranges are set to correspond to different second comfort scores. Lower skin conductance values ​​are associated with higher second comfort scores, while higher skin conductance values ​​are associated with lower second comfort scores.

[0097] Step S422: performing weighted calculation on the first comfort score and the second comfort score according to a preset weight to obtain a comfort score.

[0098] In this embodiment, the first comfort score and the second comfort score are weightedly calculated according to preset weights to obtain a final comfort score. The preset weights are determined based on the importance of blinking frequency and skin conductance in reflecting the wearer's comfort.

[0099] By comprehensively considering the two physiological signals of blinking frequency and skin conductance, determining their corresponding comfort scores respectively, and then performing weighted calculations, the wearer's comfort state can be evaluated more comprehensively and accurately. Compared with the single physiological signal evaluation method, this method can reduce the errors caused by individual differences and signal fluctuations, and improve the reliability of the comfort score.

[0100] Step S50: update the optical parameter prediction neural network model based on the comfort score, and return to the step of collecting the wearer's target biometric data through the biometric collection module.

[0101] In this embodiment, after obtaining the comfort score, the head-mounted display device analyzes and processes it. If the analysis results indicate a high comfort score, this indicates that the target optical parameters generated by the current optical parameter prediction neural network model closely match the wearer's actual needs, meeting the user's visual expectations and not causing noticeable discomfort during extended wear. This positive feedback demonstrates that the model has achieved a high degree of accuracy in capturing subtle differences in the user's biometrics and predicting optimal optical parameters. Therefore, the device takes further action to store the collected biometric data (such as pupillary distance, corneal curvature, eye movement trajectory, etc.), target optical parameters (such as focal length, light field distribution, color temperature adjustment, etc.), and the user's comfort score as a complete training sample in the device's database. These samples not only record the user's current physiological state and preferences, but also reflect the output performance of the optical parameter prediction neural network model under specific conditions, providing data resources for the continuous optimization of subsequent models.

[0102] Subsequently, these newly added training samples will be incorporated into the training process of the optical parameter prediction neural network model. Through machine learning algorithms, the parameters and structure of the optical parameter prediction neural network model will be continuously iterated and optimized, so that in future uses, the target optical parameters that meet the user's personalized needs can be more accurately predicted and generated. This process is an iterative and continuous improvement process aimed at continuously improving the comfort and performance of the head-mounted display device.

[0103] On the contrary, if the analysis results show that the comfort score is low, this indicates that the current optical parameter prediction neural network model has deviations when outputting the target optical parameters, and fails to fully consider the wearer's biometric differences or the needs of specific usage scenarios. The device will use this comfort score as a direct feedback signal and backpropagate it to each layer of the optical parameter prediction neural network model. By adjusting the weight parameters of each layer of the neural network, it guides it to learn a more accurate optical parameter generation strategy that is more in line with user needs.

[0104] After adjusting the weight parameters of the optical parameter prediction neural network model, the device will re-collect the wearer's target biometric data and input it into the updated and adjusted optical parameter prediction neural network model to generate new target optical parameters. This process may be repeated multiple times until the newly generated optical parameters can obtain a higher comfort score in subsequent user tests. Through this continuous iteration and continuous optimization method, the head-mounted display device can gradually approach the limits of user needs and provide users with a more comfortable and personalized visual experience.

[0105] Furthermore, in a feasible implementation, the optical parameter prediction neural network model is a GAN model, and the step of updating the GAN model based on the comfort score in the above step S50 includes steps S51 and S52: Step S51, determining the loss function of the GAN model according to the comfort score; In this embodiment, the loss function of the GAN model can be obtained by weighted calculation based on the comfort score.

[0106] Specifically, the loss function L combines adversarial loss with user rating weighting: ; L adv : Adversarial Loss, which measures the probability that the generator output parameters are detected by the discriminator.

[0107] L score : User rating loss (Comfort Loss), which quantifies the degree of matching between the generated parameters and the user's comfort.

[0108] λ1, λ2: Weight coefficients used to balance adversarial training and user feedback. The default values ​​are λ1=0.7 and λ2=0.3, which need to be adjusted based on the validation set.

[0109] Step S52: Update the weights of the generator in the GAN model based on the preset gradient descent algorithm and loss function.

[0110] In this embodiment, a preset gradient current algorithm is further used in combination with a loss function to update the weights of the generator in the GAN model, and the model is gradually adjusted to the optimal state.

[0111] Specifically, the generator weights can be updated using mini-batch gradient descent (Mini-batch SGD): ; θ G : Trainable parameters of the generator; η: Learning rate, which controls the parameter update step size. The initial value of η is set to 0.0002, and it decays exponentially with the number of training rounds: the initial learning rate η0=0.0002; ▽ θG : Gradient of the loss function with respect to the generator parameters.

[0112] S, Y: User rating and currently generated optical parameters.

[0113] In a feasible implementation manner, before the above step S50, the optical adjustment method further includes steps B10 to B20: Step B10, determining whether the comfort score is less than a first preset score; It should be noted that the first preset score refers to a baseline value or threshold set in the optical adjustment system of the head-mounted display (HMD) device, used to assess whether the wearer's visual comfort meets acceptable standards. This score is based on a series of predefined rules and standards, or an ideal comfort level derived through extensive user testing and data analysis. Specifically, the first preset score is a numerical standard that represents the minimum visual comfort level that the wearer should achieve when using the HMD device. It is typically determined by the device manufacturer based on extensive user experience data, physiological research, and the needs of specific application scenarios. This score serves as a basis for determining whether the optical parameter prediction neural network model needs to be updated. If the wearer's target comfort score is lower than the first preset score, it indicates that the current optical settings do not fully meet the user's visual needs, and the optical parameter prediction neural network model has room for improvement. Conversely, if the target comfort score is equal to or higher than the first preset score, it indicates that the current settings already provide a good user experience, and no further updates or adjustments to the optical parameter prediction neural network model are required.

[0114] By introducing the concept of a first preset score, this implementation intelligently determines when to initiate updates to the optical parameter prediction neural network model while ensuring a positive user experience. This improves the model's accuracy and ensures an optimal visual experience for users. This mechanism not only helps reduce unnecessary resource consumption but also significantly improves user satisfaction and overall device performance.

[0115] Step B20: When it is determined that the target comfort score is less than the first preset score, executing the step of updating the optical parameter prediction neural network model based on the comfort score.

[0116] In this embodiment, if the comfort score is determined to be less than the first preset score, it means that the wearer's current visual experience is not ideal and there may be a significant VAC problem. At this time, the system automatically initiates the update process and enters step S50 to update the optical parameter prediction neural network model based on the comfort score.

[0117] This implementation adopts a judge-before-act mechanism, which not only improves the system's response efficiency and avoids unnecessary resource consumption, but also ensures that corresponding model updates are only performed when there is a real need to improve the wearer experience, helping to maintain the stability of device performance and long-term user satisfaction.

[0118] This embodiment introduces a scoring module to perform real-time evaluation of the wearer's comfort level, and dynamically updates the optical parameter prediction neural network model based on the evaluation results (i.e., the comfort score), thereby achieving personalized prediction of target optical parameters. This ensures that the target optical parameters predicted by the optical parameter prediction neural network model are more closely aligned with the wearer's target biometric data, thereby effectively reducing the visual convergence-accommodation conflict (VAC effect) in the head-mounted display device and improving the user's wearing experience.

[0119] Furthermore, in a feasible implementation manner, after the above step B10, the optical adjustment method further includes steps B30 to B40: Step B30, calibrating target optical parameters based on the comfort score; In this embodiment, when it is determined that the current optical settings fail to fully meet the wearer's visual comfort needs and that a certain degree of VAC effect or other discomfort factors exist, the system can use a feedback control algorithm (such as PID control, gradient descent method, etc.) to fine-tune the original target optical parameters based on the degree of deviation between the comfort score and the ideal score, thereby generating a set of optimized optical parameters that are more consistent with the wearer's subjective feelings and physiological state, i.e., the calibrated target optical parameters.

[0120] Step B40: adjusting the control voltage of the liquid lens array according to the calibrated target optical parameters to adjust the curvature of the liquid lens array.

[0121] After completing the calibration of the target optical parameters, this embodiment can convert these optimized parameters (i.e., the calibrated target optical parameters) into specific control instructions and input them into the liquid lens array's driver module. Each lens unit in the liquid lens array has a variable curvature characteristic, and its curvature can be dynamically controlled by applying different voltages. Therefore, this embodiment calculates the corresponding control voltage value based on the calibrated target optical parameters and applies it to the corresponding liquid lens unit, thereby changing the curvature distribution of the lens surface.

[0122] This implementation implements a closed-loop feedback mechanism from "perceived comfort" to "physical optical adjustment": the wearer's comfort score is converted into actual optical parameter changes, which are ultimately reflected in the physical morphology adjustment of the liquid lens array, making the virtual image more closely aligned with the user's visual focus, effectively alleviating the VAC effect, and improving display quality and user experience.

[0123] In summary, the target optical parameters are calibrated based on the comfort score in step B30, and the calibration results are applied to the actual adjustment of the liquid lens array in step B40. This embodiment constructs a dynamic adaptive optical adjustment mechanism that integrates score feedback and physical adjustment, significantly improving the visual adaptability and user satisfaction of the head-mounted display device in complex application scenarios.

[0124] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the optical adjustment method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0125] The present application provides a head-mounted display device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the optical adjustment method of the above-mentioned embodiment 1.

[0126] Reference below Figure 5 , which shows a schematic diagram of the structure of a head-mounted display device suitable for implementing the embodiments of the present application. The head-mounted display device in the embodiments of the present application may include, but is not limited to, head-mounted display devices such as Mixed Reality (MR) devices (e.g., MR glasses or MR helmets), Augmented Reality (AR) devices (e.g., AR glasses or AR helmets), Virtual Reality (VR) devices (e.g., VR glasses or VR helmets), Extended Reality (XR) devices, or some combination thereof.

[0127] like Figure 5As shown, the head-mounted display device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the head-mounted display device. The processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speakers, and vibrator; storage devices 1003 including, for example, magnetic tape and hard disk; and communication devices 1009. The communication device 1009 can allow the head-mounted display device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a head-mounted display device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0128] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0129] The head-mounted display device provided in this application, employing the optical adjustment method described in the aforementioned embodiment, can address the technical problem of reducing the VAC effect of head-mounted display devices. Compared to the prior art, the head-mounted display device provided in this application achieves the same beneficial effects as the optical adjustment method described in the aforementioned embodiment. Other technical features of this head-mounted display device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0130] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0131] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0132] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the optical adjustment method in the above-mentioned embodiment.

[0133] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0134] The computer-readable storage medium may be included in the head-mounted display device, or may exist independently without being incorporated into the head-mounted display device.

[0135] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a head-mounted display device, the head-mounted display device: collects target biometric data of the wearer through a biometric collection module; inputs the target biometric data into a preset optical parameter prediction neural network model to predict target optical parameters, wherein the optical parameter prediction neural network model is trained using the biometric data as samples and the optical parameters as labels; and adjusts the control voltage of the liquid lens array based on the target optical parameters to adjust the target curvature of the liquid lens array.

[0136] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0138] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0139] The computer-readable storage medium provided herein stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned optical adjustment method, thereby solving the technical problem of reducing the VAC effect of head-mounted display devices. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided herein are similar to those of the optical adjustment method provided in the aforementioned embodiments and are not further elaborated here.

[0140] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned optical adjustment method when executed by a processor.

[0141] The computer program product provided in this application can solve the technical problem of reducing the VAC effect of head-mounted display devices. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the optical adjustment method provided in the above embodiment, and will not be repeated here.

[0142] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. An optical adjustment method, characterized in that: The optical adjustment method is applied to a head-mounted display device, which includes: a biometric feature acquisition module and a liquid lens array; the method includes: Collecting target biometric data of the wearer through the biometric collection module; Inputting the target biometric data into a preset optical parameter prediction neural network model to predict target optical parameters, wherein the optical parameter prediction neural network model is trained using the biometric data as samples and the optical parameters as labels; A control voltage of the liquid lens array is adjusted based on the target optical parameter to adjust the curvature of the liquid lens array.

2. The optical adjustment method according to claim 1, wherein: The target optical parameters include: focal length compensation value, lens curvature and dispersion compensation coefficient; The step of adjusting the control voltage of the liquid lens array based on the target optical parameters comprises: Calculating a curvature correction term according to the focal length compensation value and the dispersion compensation coefficient; Adding the lens curvature and the curvature correction term to obtain a target curvature of the liquid lens array; Calculating a curvature difference between an initial curvature of the liquid lens array and a target curvature; Calculating a target voltage based on the curvature difference and a preset curvature-voltage coefficient; The control voltage is adjusted according to the target voltage.

3. The optical adjustment method according to claim 1, wherein: Before the step of inputting the target biometric data into a preset optical parameter prediction neural network model, the method further includes: If the target biometric data does not conform to a preset normal range, recollect the target biometric data; The target biometric data is input into a preset convolutional network to obtain vectorized target biometric data.

4. The optical adjustment method according to claim 1, wherein: The head-mounted display device further includes: a score acquisition module; After the step of adjusting the control voltage of the liquid lens array based on the target optical parameters, the method further includes: Obtaining a comfort score through the score acquisition module; The optical parameter prediction neural network model is updated based on the comfort score, and the step of collecting the wearer's target biometric data through the biometric collection module is returned to be executed.

5. The optical adjustment method according to claim 4, wherein: The score acquisition module includes: a physiological signal acquisition unit; The step of obtaining the comfort score through the score obtaining module includes: collecting target physiological signals of the wearer through the physiological signal collecting unit; The comfort score corresponding to the target physiological signal is determined according to a preset mapping relationship.

6. The optical adjustment method according to claim 5, wherein: The target physiological signals include: blink frequency and skin conductance; The step of determining the comfort score corresponding to the target physiological signal according to the preset mapping relationship includes: Determining a first comfort score corresponding to the blinking frequency and a second comfort score corresponding to the skin conductance according to a preset mapping relationship, wherein the blinking frequency is negatively correlated with the first comfort score, and the skin conductance is negatively correlated with the second comfort score; The comfort score is obtained by performing weighted calculation on the first comfort score and the second comfort score according to a preset weight.

7. The optical adjustment method according to claim 4, wherein: The optical parameter prediction neural network model is a GAN model, and the step of updating the optical parameter prediction neural network model based on the comfort score includes: Determining a loss function of the GAN model according to the comfort score; The weights of the generator in the GAN model are updated based on a preset gradient descent algorithm and the loss function.

8. The optical adjustment method according to claim 1, wherein: The biometric feature acquisition module includes: a camera unit and an eye movement detection unit; the target biometric feature data includes: pupil distance, corneal curvature, gaze point coordinates and eye movement speed; The step of collecting target biometric data of the wearer through the biometric collection module includes: Acquiring an eye image of the wearer through the camera unit; acquiring the pupil distance and corneal curvature of the wearer based on the eyeball image; The wearer's gaze point coordinates and eye movement speed are obtained through the eye movement detection unit.

9. A head-mounted display device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the optical adjustment method according to any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the optical adjustment method according to any one of claims 1 to 8 are implemented.

Images