An eye movement tracking zoom control system based on visual-electrophysiological multi-modal and zoom glasses

Abstract

This invention discloses a vision-electrophysiological multimodal eye-tracking zoom control system and zoom glasses, relating to the field of zoom glasses technology. The zoom optical element has at least one control area. The multimodal sensing unit includes a vision detection module, an EOG detection module, and an ambient light sensing module, used to collect the wearer's eye state information, periocular potential information, and ambient light information, respectively, when the glasses are worn. A cross-modal adaptive calibration unit is used to calibrate the eye state information using periocular potential information when the ambient light information does not meet preset lighting conditions. The focus control unit is used to calculate the eye's gaze direction and gaze distance based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus using the focus control parameters. This invention improves zoom performance by coordinating visual signals and EOG signals.

Description

Technical Field

[0001] This invention relates to the field of zoom glasses technology, and more particularly to an eye-tracking zoom control system based on visual-electrophysiological multimodality and zoom glasses. Background Technology

[0002] Zoom glasses are optical devices that automatically adjust the focal length of the lenses according to the wearer's visual needs. Their core function is to achieve clear imaging at different viewing distances through variable-focus optical elements. Compared to traditional bifocal or multifocal lenses, electronic zoom glasses typically use liquid crystal variable-focus lenses. By applying voltage to control electrodes, the orientation of liquid crystal molecules is changed, thereby achieving continuous focal length adjustment and providing a more natural visual transition effect in different distance scenarios.

[0003] However, existing electronic zoom glasses still face key technical bottlenecks in practical applications. First, in terms of distance perception, existing solutions mostly rely on single-vision eye-tracking methods. Single-vision eye-tracking methods are highly dependent on ambient lighting conditions and are prone to accuracy degradation or even failure in extremely dark or brightly lit environments. Second, although traditional eye-tracking detection methods based on electrooculography (EOG) have the advantages of low power consumption and insensitivity to lighting, their spatial resolution is low, making it difficult to directly use them for high-precision gaze direction and distance estimation.

[0004] Therefore, how to achieve stable and continuous estimation of the wearer's gaze direction and gaze distance without relying on external ranging devices, and maintain the accuracy and robustness of eye tracking in complex lighting environments, has become a key technical problem that urgently needs to be solved in the field of smart zoom glasses. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an eye-tracking zoom control system and zoom glasses based on visual-electrophysiological multimodal approaches. This solves the technical problems of existing zoom glasses, such as strong dependence on external ranging devices during distance perception, insufficient stability of eye movement detection under complex lighting conditions, and difficulty in balancing the continuity and accuracy of focusing response.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides an eye-tracking zoom control system based on visual-electrophysiological multimodal approaches, comprising:

[0008] A variable-focus optical element is disposed in the optical imaging path of the glasses and has at least one control area;

[0009] The multimodal sensing unit includes a visual detection module, an EOG detection module, and an ambient light sensing module, which are used to collect information on the wearer's eye state, periocular potential, and ambient light when the glasses are worn.

[0010] A cross-modal adaptive calibration unit is used to calibrate the eye state information using the periocular potential information when the ambient light information does not meet the preset light conditions.

[0011] The focus control unit is used to calculate the gaze direction and gaze distance of the eye based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus through the focus control parameters.

[0012] Optionally, when the ambient light information meets preset lighting conditions, the periocular potential information is calibrated using the eye state information, including:

[0013] When the ambient light information meets the preset lighting conditions, acquire the eye state information and periocular potential information of the wearer when gazing at multiple calibration points;

[0014] A bioelectrical model of both eyes is established, and the model parameters of the bioelectrical model are fitted based on the eye state information and periocular potential information of multiple calibration points to complete the calibration of the bioelectrical model;

[0015] The eyeball state information includes the horizontal rotation angle of the left eyeball. Horizontal rotation of the right eyeball Horizontal rotation of the eyeball Vertical angle of eyeball Eye radius R, interpupillary distance b;

[0016] The periocular potential information includes the potentials of multiple electrodes disposed around the eye.

[0017] Optionally, the electrodes are respectively disposed on the left nose pad, right nose pad, left temple, right temple and any earlobe of the glasses, or disposed on the left nose pad, right nose pad, any temple and any earlobe of the glasses.

[0018] Optionally, establishing a bioelectrical model of both eyeballs includes:

[0019] If both eyeballs are considered as bioelectric dipoles, with the cornea and retina as the positive and negative poles respectively, then the bioelectric model of the two eyeballs is as follows:

[0020]

[0021] In the formula, Let be the potential of the i-th electrode when viewing the j-th calibration point. Let be the distances from the cornea and retina of the right eye to the i-th electrode when fixating on the j-th calibration point. Let be the distances from the cornea and retina of the left eye to the i-th electrode when fixating on the j-th calibration point. These are all model parameters for the i-th electrode.

[0022] Optionally, the model parameters for fitting the bioelectrical model based on eye state information and periocular potential information from multiple calibration points include:

[0023] Distance is calculated based on eye state information from multiple calibration points. ;

[0024] By setting any one electrode as the reference electrode, the potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. And based on the potential difference Construct the observation vector y; where, The potential of the reference electrode when viewing the j-th calibration point, Let be the potential difference between the i-th electrode and the reference electrode when viewing the j-th calibration point;

[0025] Design matrix X to represent different potential differences. As rows of the matrix X, with a row number of (I-1)*J, the potential difference is... Corresponding model parameters coefficients of model parameters As columns, the number of columns is 4I; I and J are the number of electrodes and the number of calibration points, respectively;

[0026] The coefficients of the model parameters As the element values ​​of matrix X, the model parameters As an unknown matrix The element values, combined with the observation vector y, are used to construct an overdetermined system of equations: ;

[0027] The overdetermined system of equations was solved using the least squares method. Solve the problem based on the unknown matrix of the solution. Reconstructed potential difference Calculate the root mean square error before and after potential difference reconstruction. When the root mean square error is less than the error threshold, the calibrated unknown matrix is ​​obtained. The model parameters of the bioelectric model are then fitted.

[0028] Optionally, the distance is calculated based on eye state information from multiple calibration points. include:

[0029] The distance between the cornea and the retina and the center of the eyeball is equal to the radius R of the eyeball. A reference three-dimensional coordinate system is constructed with the center positions of the left and right eyeballs as the origin. The coordinates of the right eyeball cornea, the right eyeball retina, the left eyeball cornea, and the left eyeball retina are calculated based on the eyeball state information when looking at the j-th calibration point.

[0030] Obtain the spatial coordinates of the i-th electrode and transform them into the reference three-dimensional coordinate system. Combine the right eye corneal coordinates, the right eye retinal coordinates, the left eye corneal coordinates, and the left eye retinal coordinates to calculate the distance. .

[0031] Optionally, calibrating the ocular state information using the periocular potential information includes:

[0032] The potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. ,in, The potential of the reference electrode when fixating on the current fixation point. Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point; Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point;

[0033] The potential difference of the reference electrode Use the pre-built mapping table as an index to obtain the gaze coordinates of the current gaze point;

[0034] Calculate the fitted value of the eye state information based on the coordinates of the current gaze point;

[0035] The final value of the eyeball state information is obtained by weighted summing of the original value and the fitted value.

[0036] The mapping table is a mapping relationship between the potential difference of each electrode relative to the reference electrode and the gaze coordinates of the gaze point, constructed based on the visual detection module and the calibrated bioelectric model.

[0037] Optionally, the weighted summation of the original and fitted values ​​of the eye state information includes:

[0038]

[0039] In the formula, These are weighting coefficients. These represent the original, fitted, and final values ​​of the eyeball state information, respectively. Weighting coefficients The degree to which the ambient lighting information deviates from the preset lighting conditions is negatively correlated.

[0040] Optionally, the visual detection module includes:

[0041] Image sensor used to acquire images of the wearer's eye area;

[0042] A depth sensor is used to collect the depth value of each pixel in the wearer's eye area;

[0043] Event vision sensor, used to capture event streams of the wearer's eye movements;

[0044] The processing module is used to obtain the output results of the image sensor, the depth sensor and the event vision sensor to calculate the wearer's eye state information.

[0045] In a second aspect, the present invention provides zoom glasses, including the eye-tracking zoom control system based on visual-electrophysiological multimodal as described above.

[0046] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0047] This invention provides an eye-tracking zoom control system and zoom glasses based on visual-electrophysiological multimodal approaches. By constructing a collaborative working mechanism between visual signals and EOG signals, the visual detection module calibrates and constrains the EOG signals, and the EOG detection module compensates for eye-tracking estimation results under visual degradation conditions. This significantly improves the robustness and continuity of the system in complex environments while ensuring spatial accuracy. Attached Figure Description

[0048] Figure 1 This is an external view structural diagram of the intelligent zoom LCD glasses system provided by the present invention;

[0049] Figure 2 This is an internal view structure diagram of the intelligent zoom LCD glasses system provided by the present invention;

[0050] The diagram is marked as follows:

[0051] 1. Variable focus optical element; 2. Modal sensing unit; 201. Visual inspection module; 202. EOG detection module; 203. Ambient light sensing module; 3. Cross-modal adaptive calibration unit; 4. Focusing control unit. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0053] Example 1

[0054] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides an eye-tracking zoom control system based on visual-electrophysiological multimodal approaches, comprising:

[0055] 1. A variable focus optical element 1 is disposed in the optical imaging path of the eyeglasses and has at least one control area.

[0056] 2. Modal sensing unit 2, including visual detection module 201, EOG detection module 202 and ambient light sensing module 203, are used to collect the wearer's eye state information, periocular potential information and ambient light information when the glasses are worn.

[0057] 3. Cross-modal adaptive calibration unit 3 is used to calibrate the eye state information by means of periocular potential information when the ambient light information does not meet the preset light conditions.

[0058] 4. Focus control unit 4 is used to calculate the gaze direction and gaze distance of the eyeball based on the eyeball state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus through the focus control parameters.

[0059] A. Regarding the variable focus optical element 1, specifically in this embodiment, the variable focus optical element 1 is treated as a whole as a control region or divided into multiple independent control regions, and the boundaries of the control regions are circular or approximately circular continuous curves. In other optional embodiments, those skilled in the art can set the control region to any closed-loop shape as needed. In this embodiment, setting it to a circular or approximately circular continuous curve can reduce liquid crystal orientation distortion, electric field discontinuity, or optical performance fluctuations caused by boundary sharp edges, thereby improving the focusing stability and optical consistency of the liquid crystal lens.

[0060] The embodiments of the present invention provide a choice between a single control region and multiple independent control regions. A single control region only requires one drive channel, which has low hardware and manufacturing costs, but the overall adjustment power consumption is high and lacks specificity. Multiple independent control regions require a corresponding number of drive channels, which has higher hardware and manufacturing costs, but because the area of ​​each control region is reduced, the power consumption is lower, and the focus on the gaze direction is stronger, resulting in better visual effects.

[0061] B. Regarding the visual detection module 201, specifically in this embodiment, it includes an image sensor (RGB), a depth sensor (ToF), an event vision sensor (DVS / EVS), and a processing module. The image sensor is used to acquire images of the wearer's eye region; the depth sensor is used to acquire the depth value of each pixel in the wearer's eye region; and the event vision sensor is used to acquire the event stream of the wearer's eye movements. The processing module is used to obtain the output results of the image sensor, depth sensor, and event vision sensor to calculate the wearer's eye state information.

[0062] Eye status information includes, but is not limited to, the horizontal rotation angle of the left eyeball. Horizontal rotation of the right eyeball Horizontal rotation of the eyeball Vertical angle of eyeball Eyeball radius R, interpupillary distance between left and right eyes b.

[0063] C. Regarding the EOG detection module 202, specifically in this embodiment, it includes multiple electrodes, which are respectively disposed on the left nose pad, right nose pad, left temple, right temple of the glasses, and any earlobe. The electrode on the earlobe is used as a reference electrode. This design can effectively cancel common-mode interference caused by head movement or environment, thereby ensuring from an electrical perspective that the captured "bi-eye angle difference" is pure, which is the physical basis for calculating the accurate distance.

[0064] In a low-cost, minimal configuration, multiple electrodes are placed on the left and right nose pads, any temple, and any earlobe of the glasses.

[0065] Periocular potential information includes, but is not limited to, the potentials of multiple electrodes placed around the eye.

[0066] D. Regarding the ambient light sensing module 203, in this specific embodiment, it is set as a light sensor to obtain the ambient light intensity, thereby determining whether the current environment is extremely dark or has strong light interference.

[0067] E. Regarding the cross-modal adaptive calibration unit 3, specifically in this embodiment:

[0068] E.1. When the ambient light information meets the preset lighting conditions, the periocular potential information is calibrated using eye state information, including:

[0069] When the ambient light information meets the preset lighting conditions, acquire the eye state information and periocular potential information of the wearer when looking at multiple calibration points;

[0070] A bioelectrical model of both eyes was established, and the model parameters of the bioelectrical model were fitted based on the eye state information and periocular potential information of multiple calibration points to complete the calibration of the bioelectrical model.

[0071] E.1.1 Establishing a bioelectrical model of both eyes includes:

[0072] If we consider both eyes as bioelectric dipoles, with the cornea and retina as the positive and negative poles respectively, then the bioelectric model of both eyes is as follows:

[0073]

[0074] In the formula, Let be the potential of the i-th electrode when viewing the j-th calibration point. Let be the distances from the cornea and retina of the right eye to the i-th electrode when fixating on the j-th calibration point. Let be the distances from the cornea and retina of the left eye to the i-th electrode when fixating on the j-th calibration point. These are all model parameters for the i-th electrode.

[0075] E.1.2 The model parameters for fitting the bioelectrical model based on eye state information and periocular potential information from multiple calibration points include:

[0076] Distance is calculated based on eye state information from multiple calibration points. :

[0077] By setting any one electrode as the reference electrode, the potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. And based on potential difference Construct the observation vector y; where, The potential of the reference electrode when viewing the j-th calibration point, Let be the potential difference between the i-th electrode and the reference electrode when viewing the j-th calibration point;

[0078] Design matrix X to represent different potential differences. As rows of matrix X, with row number (I-1)*J, the potential difference is... Corresponding model parameters coefficients of model parameters As columns, the number of columns is 4I; I and J are the number of electrodes and the number of calibration points, respectively;

[0079] The coefficients of the model parameters As the element values ​​of matrix X, the model parameters As an unknown matrix The element values, combined with the observation vector y, are used to construct an overdetermined system of equations: ;

[0080] The least squares method is used to solve the overdetermined system of equations. Solve the problem based on the unknown matrix of the solution. Reconstructed potential difference Before calculating the potential difference reconstruction ( )back( ) root mean square error :

[0081]

[0082] When the root mean square error When the error is less than the threshold, the calibrated unknown matrix is ​​obtained. Complete the fitting of model parameters for the bioelectric model.

[0083] Taking an electrode count of I=5 as an example, the potential difference between the first electrode and the first calibration point for:

[0084]

[0085] The model parameters and their coefficients corresponding to the first electrode and the first calibration point are shown in Table 1.

[0086] Table 1: Model parameters and coefficients corresponding to the first electrode and the first calibration point

[0087]

[0088] Similarly, the row of the second electrode has non-zero coefficients only in columns 5-8 and 17-20, the row of the third electrode has non-zero coefficients in columns 9-12 and 17-20, and the row of the fourth electrode has non-zero coefficients in columns 13-16 and 17-20.

[0089] Construct an unknown matrix using the model parameters and coefficients for all electrodes except the reference electrode. And matrix X.

[0090] E.1.3 Calculating distance based on eye state information from multiple calibration points include:

[0091] The distance between the cornea and retina and the center of the eyeball is equal to the radius R of the eyeball. A reference three-dimensional coordinate system is constructed with the center positions of the left and right eyeballs as the origin. The coordinates of the right eyeball cornea, right eyeball retina, left eyeball cornea, and left eyeball retina are calculated based on the eyeball state information when looking at the j-th calibration point.

[0092] The center coordinates of the right and left eyeballs They are respectively:

[0093]

[0094]

[0095] Calculate the corneal coordinates of the right eye :

[0096]

[0097] Calculate the retinal coordinates of the right eye :

[0098]

[0099] Calculate the corneal coordinates of the left eye :

[0100]

[0101] Calculate the coordinates of the left eye's retina :

[0102]

[0103] Obtain the spatial coordinates of the i-th electrode and transform them into a reference three-dimensional coordinate system. Combine the coordinates of the right eye cornea, right eye retina, left eye cornea, and left eye retina to calculate the distance. .

[0104] Let the coordinates of the i-th electrode be... Then the distance for:

[0105]

[0106]

[0107]

[0108] E.1.4. Calibration of ocular state information using periocular potential information includes:

[0109] The potential difference between each electrode and the reference electrode is calculated based on periocular potential information. ,in, The potential of the reference electrode when fixating on the current fixation point. Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point; Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point;

[0110] The potential difference of the reference electrode Use the pre-built mapping table as an index to obtain the gaze coordinates of the current gaze point;

[0111] Calculate the fitted value of the eye state information based on the coordinates of the current gaze point;

[0112] Let the gaze coordinates of the current gaze point be... The fitted value is calculated as follows:

[0113]

[0114]

[0115]

[0116]

[0117] In the formula, These are the horizontal rotation angles of the eyeball. Vertical angle of eyeball Horizontal angle of left eyeball Horizontal rotation of the right eyeball The fitted value is given by b, where b is the interpupillary distance between the left and right eyes.

[0118] The final value of the eye state information is obtained by weighted summing of the original and fitted values. This weighted summation includes:

[0119]

[0120] In the formula, These are weighting coefficients. These represent the original, fitted, and final values ​​of the eyeball state information, respectively. Weighting coefficients The degree to which ambient lighting information deviates from preset lighting conditions is negatively correlated;

[0121] The mapping table is a mapping relationship between the potential difference of each electrode relative to the reference electrode and the gaze coordinates of the gaze point, constructed based on the visual detection module 201 and the calibrated bioelectric model.

[0122] Specifically, in this embodiment, the mapping relationship can be constructed using models such as Transformer, RNN, SVR, Kalman filter, and MLP.

[0123] In constructing the mapping relationship, preprocessing techniques such as bandpass filtering, power frequency denoising, drift elimination, and signal normalization can be applied to the periocular potential information to provide cleaner data support for subsequent processing. Simultaneously, the periocular potential information and the gaze coordinates of the fixation point are time-aligned to avoid temporal misalignment that could prevent model convergence.

[0124] F. Regarding the focusing control unit 4, specifically in this embodiment:

[0125] F.1 Calculate the gaze direction and gaze distance based on eye state information:

[0126] Based on the horizontal rotation angle and center position coordinates of the eyeball Calculate the eye's fixation distance and determine the eye's fixation direction based on the horizontal and vertical rotation angles of the eye.

[0127] Eye gaze distance for:

[0128]

[0129]

[0130] In the formula, For a moment, These refer to the horizontal rotation angles of the left and right eyeballs, respectively. The distance between the left and right eyeballs. ...

[0131] This invention determines the distance of the object being viewed by the wearer based on the convergence angle relationship formed between the horizontal turning angles of the left and right eyes and the distance between the centers of the left and right eyeballs. The entire calculation is efficient and accurate, which can effectively improve the efficiency and precision of the entire focusing process.

[0132] F.2. The focus control parameters generated based on the gaze distance include:

[0133] The mapping relationships between gaze distance and equivalent focal length, equivalent focal length and applied voltage signal, and applied voltage signal and focus control parameters are constructed; the focus control parameters corresponding to gaze distance are obtained through layer-by-layer mapping.

[0134] Example 2

[0135] This invention provides a zoom glasses system, including the eye-tracking zoom control system based on visual-electrophysiological multimodal as described in Embodiment 1 above.

[0136] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0140] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A vision-electrophysiological multimodal eye-tracking zoom control system, characterized in that, include: A variable-focus optical element is disposed in the optical imaging path of the glasses and has at least one control area; The multimodal sensing unit includes a visual detection module, an EOG detection module, and an ambient light sensing module, which are used to collect information on the wearer's eye state, periocular potential, and ambient light when the glasses are worn. A cross-modal adaptive calibration unit is used to calibrate the eye state information using the periocular potential information when the ambient light information does not meet the preset light conditions. The focus control unit is used to calculate the gaze direction and gaze distance of the eye based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus through the focus control parameters.

2. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 1, characterized in that, When the ambient light information meets the preset light conditions, the periocular potential information is calibrated using the eye state information, including: When the ambient light information meets the preset lighting conditions, acquire the eye state information and periocular potential information of the wearer when gazing at multiple calibration points; A bioelectrical model of both eyes is established, and the model parameters of the bioelectrical model are fitted based on the eye state information and periocular potential information of multiple calibration points to complete the calibration of the bioelectrical model; The eyeball state information includes the horizontal rotation angle of the left eyeball. Horizontal rotation of the right eyeball Horizontal rotation of the eyeball Vertical angle of eyeball Eye radius R, interpupillary distance b; The periocular potential information includes the potentials of multiple electrodes disposed around the eye.

3. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 2, characterized in that, The electrodes are respectively disposed on the left nose pad, right nose pad, left temple, right temple and any earlobe of the glasses, or disposed on the left nose pad, right nose pad, any temple and any earlobe of the glasses.

4. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 2, characterized in that, The establishment of the bioelectrical model of the two eyeballs includes: If both eyeballs are considered as bioelectric dipoles, with the cornea and retina as the positive and negative poles respectively, then the bioelectric model of the two eyeballs is as follows: ； In the formula, Let be the potential of the i-th electrode when viewing the j-th calibration point. Let be the distances from the cornea and retina of the right eye to the i-th electrode when fixating on the j-th calibration point. Let be the distances from the cornea and retina of the left eye to the i-th electrode when fixating on the j-th calibration point. These are all model parameters for the i-th electrode.

5. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 4, characterized in that, The model parameters for fitting the bioelectrical model based on eye state information and periocular potential information from multiple calibration points include: Distance is calculated based on eye state information from multiple calibration points. ; By setting any one electrode as the reference electrode, the potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. And based on the potential difference Construct the observation vector y; where, The potential of the reference electrode when viewing the j-th calibration point, Let be the potential difference between the i-th electrode and the reference electrode when viewing the j-th calibration point; Design matrix X to represent different potential differences. As rows of the matrix X, with a row number of (I-1)*J, the potential difference is... Corresponding model parameters coefficients of model parameters As columns, the number of columns is 4I; I and J are the number of electrodes and the number of calibration points, respectively; The coefficients of the model parameters As the element values ​​of matrix X, the model parameters As an unknown matrix The element values, combined with the observation vector y, are used to construct an overdetermined system of equations: ; The overdetermined system of equations was solved using the least squares method. Solve the unknown matrix based on the solution. Reconstructed potential difference Calculate the root mean square error before and after potential difference reconstruction. When the root mean square error is less than the error threshold, the calibrated unknown matrix is ​​obtained. The model parameters of the bioelectric model are then fitted.

6. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 5, characterized in that, The distance is calculated based on eye state information from multiple calibration points. include: The distance between the cornea and the retina and the center of the eyeball is equal to the radius R of the eyeball. A reference three-dimensional coordinate system is constructed with the center positions of the left and right eyeballs as the origin. The coordinates of the right eyeball cornea, the right eyeball retina, the left eyeball cornea, and the left eyeball retina are calculated based on the eyeball state information when looking at the j-th calibration point. Obtain the spatial coordinates of the i-th electrode and transform them into the reference three-dimensional coordinate system. Combine the right eye corneal coordinates, the right eye retinal coordinates, the left eye corneal coordinates, and the left eye retinal coordinates to calculate the distance. .

7. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 5, characterized in that, The calibration of the eye state information using the periocular potential information includes: The potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. ,in, The potential of the reference electrode when fixating on the current fixation point. Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point; Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point; The potential difference of the reference electrode Use the pre-built mapping table as an index to obtain the gaze coordinates of the current gaze point; Calculate the fitted value of the eye state information based on the coordinates of the current gaze point; The final value of the eyeball state information is obtained by weighted summing of the original value and the fitted value. The mapping table is a mapping relationship between the potential difference of each electrode relative to the reference electrode and the gaze coordinates of the gaze point, constructed based on the visual detection module and the calibrated bioelectric model.

8. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 7, characterized in that, The step of weighted summing of the original and fitted values ​​of the eye state information includes: ； In the formula, These are weighting coefficients. These represent the original, fitted, and final values ​​of the eyeball state information, respectively. Weighting coefficients The degree to which the ambient lighting information deviates from the preset lighting conditions is negatively correlated.

9. The eye-tracking zoom control system based on visual-electrophysiological multimodal analysis according to claim 1, characterized in that, The visual detection module includes: Image sensor used to acquire images of the wearer's eye area; A depth sensor is used to collect the depth value of each pixel in the wearer's eye area; Event vision sensor, used to capture event streams of the wearer's eye movements; The processing module is used to obtain the output results of the image sensor, the depth sensor and the event vision sensor to calculate the wearer's eye state information.

10. A zoom lens, characterized in that, Including the eye-tracking zoom control system based on visual-electrophysiological multimodal as described in any one of claims 1-9.

Images