An eye movement tracking zoom control system based on visual-electrophysiological multi-modal and zoom glasses

Abstract

This invention discloses a vision-electrophysiological multimodal eye-tracking zoom control system and zoom glasses, relating to the field of zoom glasses technology. The zoom optical element has at least one control area. The multimodal sensing unit includes a vision detection module, an EOG detection module, and an ambient light sensing module, used to collect the wearer's eye state information, periocular potential information, and ambient light information, respectively, when the glasses are worn. A cross-modal adaptive calibration unit is used to calibrate the eye state information using periocular potential information when the ambient light information does not meet preset lighting conditions. The focus control unit is used to calculate the eye's gaze direction and gaze distance based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus using the focus control parameters. This invention improves zoom performance by coordinating visual signals and EOG signals.

Description

Technical Field

[0001] This invention relates to the field of zoom glasses technology, and more particularly to an eye-tracking zoom control system based on visual-electrophysiological multimodality and zoom glasses. Background Technology

[0002] Zoom glasses are optical devices that automatically adjust the focal length of the lenses according to the wearer's visual needs. Their core function is to achieve clear imaging at different viewing distances through variable-focus optical elements. Compared to traditional bifocal or multifocal lenses, electronic zoom glasses typically use liquid crystal variable-focus lenses. By applying voltage to control electrodes, the orientation of liquid crystal molecules is changed, thereby achieving continuous focal length adjustment and providing a more natural visual transition effect in different distance scenarios.

[0003] However, existing electronic zoom glasses still face key technical bottlenecks in practical applications. First, in terms of distance perception, existing solutions mostly rely on single-vision eye-tracking methods. Single-vision eye-tracking methods are highly dependent on ambient lighting conditions and are prone to accuracy degradation or even failure in extremely dark or brightly lit environments. Second, although traditional eye-tracking detection methods based on electrooculography (EOG) have the advantages of low power consumption and insensitivity to lighting, their spatial resolution is low, making it difficult to directly use them for high-precision gaze direction and distance estimation.

[0004] Therefore, how to achieve stable and continuous estimation of the wearer's gaze direction and gaze distance without relying on external ranging devices, and maintain the accuracy and robustness of eye tracking in complex lighting environments, has become a key technical problem that urgently needs to be solved in the field of smart zoom glasses. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide an eye-tracking zoom control system and zoom glasses based on visual-electrophysiological multimodal approaches. This solves the technical problems of existing zoom glasses, such as strong dependence on external ranging devices during distance perception, insufficient stability of eye movement detection under complex lighting conditions, and difficulty in balancing the continuity and accuracy of focusing response.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides an eye-tracking zoom control system based on visual-electrophysiological multimodal approaches, comprising:

[0008] A variable-focus optical element is disposed in the optical imaging path of the glasses and has at least one control area;

[0009] The multimodal sensing unit includes a visual detection module, an EOG detection module, and an ambient light sensing module, which are used to collect information on the wearer's eye state, periocular potential, and ambient light when the glasses are worn.

[0010] A cross-modal adaptive calibration unit is used to calibrate the eye state information using the periocular potential information when the ambient light information does not meet the preset light conditions.

[0011] The focus control unit is used to calculate the gaze direction and gaze distance of the eye based on the eye state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus through the focus control parameters.

[0012] Optionally, when the ambient light information meets preset lighting conditions, the periocular potential information is calibrated using the eye state information, including:

[0013] When the ambient light information meets the preset lighting conditions, acquire the eye state information and periocular potential information of the wearer when gazing at multiple calibration points;

[0014] A bioelectrical model of both eyes is established, and the model parameters of the bioelectrical model are fitted based on the eye state information and periocular potential information of multiple calibration points to complete the calibration of the bioelectrical model;

[0015] The eyeball state information includes the horizontal rotation angle of the left eyeball. Horizontal rotation of the right eyeball Horizontal rotation of the eyeball Vertical angle of eyeball Eye radius R, interpupillary distance b;

[0016] The periocular potential information includes the potentials of multiple electrodes disposed around the eye.

[0017] Optionally, the electrodes are respectively disposed on the left nose pad, right nose pad, left temple, right temple and any earlobe of the glasses, or disposed on the left nose pad, right nose pad, any temple and any earlobe of the glasses.

[0018] Optionally, establishing a bioelectrical model of both eyeballs includes:

[0019] If both eyeballs are considered as bioelectric dipoles, with the cornea and retina as the positive and negative poles respectively, then the bioelectric model of the two eyeballs is as follows:

[0020]

[0021] In the formula, Let be the potential of the i-th electrode when viewing the j-th calibration point. Let be the distances from the cornea and retina of the right eye to the i-th electrode when fixating on the j-th calibration point. Let be the distances from the cornea and retina of the left eye to the i-th electrode when fixating on the j-th calibration point. These are all model parameters for the i-th electrode.

[0022] Optionally, the model parameters for fitting the bioelectrical model based on eye state information and periocular potential information from multiple calibration points include:

[0023] Distance is calculated based on eye state information from multiple calibration points. ;

[0024] By setting any one electrode as the reference electrode, the potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. And based on the potential difference Construct the observation vector y; where, The potential of the reference electrode when viewing the j-th calibration point, Let be the potential difference between the i-th electrode and the reference electrode when viewing the j-th calibration point;

[0025] Design matrix X to represent different potential differences. As rows of the matrix X, with a row number of (I-1)*J, the potential difference is... Corresponding model parameters coefficients of model parameters As columns, the number of columns is 4I; I and J are the number of electrodes and the number of calibration points, respectively;

[0026] The coefficients of the model parameters As the element values ​​of matrix X, the model parameters As an unknown matrix The element values, combined with the observation vector y, are used to construct an overdetermined system of equations: ;

[0027] The overdetermined system of equations was solved using the least squares method. Solve the problem based on the unknown matrix of the solution. Reconstructed potential difference Calculate the root mean square error before and after potential difference reconstruction. When the root mean square error is less than the error threshold, the calibrated unknown matrix is ​​obtained. The model parameters of the bioelectric model are then fitted.

[0028] Optionally, the distance is calculated based on eye state information from multiple calibration points. include:

[0029] The distance between the cornea and the retina and the center of the eyeball is equal to the radius R of the eyeball. A reference three-dimensional coordinate system is constructed with the center positions of the left and right eyeballs as the origin. The coordinates of the right eyeball cornea, the right eyeball retina, the left eyeball cornea, and the left eyeball retina are calculated based on the eyeball state information when looking at the j-th calibration point.

[0030] Obtain the spatial coordinates of the i-th electrode and transform them into the reference three-dimensional coordinate system. Combine the right eye corneal coordinates, the right eye retinal coordinates, the left eye corneal coordinates, and the left eye retinal coordinates to calculate the distance. .

[0031] Optionally, calibrating the ocular state information using the periocular potential information includes:

[0032] The potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. ,in, The potential of the reference electrode when fixating on the current fixation point. Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point; Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point;

[0033] The potential difference of the reference electrode Use the pre-built mapping table as an index to obtain the gaze coordinates of the current gaze point;

[0034] Calculate the fitted value of the eye state information based on the coordinates of the current gaze point;

[0035] The final value of the eyeball state information is obtained by weighted summing of the original value and the fitted value.

[0036] The mapping table is a mapping relationship between the potential difference of each electrode relative to the reference electrode and the gaze coordinates of the gaze point, constructed based on the visual detection module and the calibrated bioelectric model.

[0037] Optionally, the weighted summation of the original and fitted values ​​of the eye state information includes:

[0038]

[0039] In the formula, These are weighting coefficients. These represent the original, fitted, and final values ​​of the eyeball state information, respectively. Weighting coefficients The degree to which the ambient lighting information deviates from the preset lighting conditions is negatively correlated.

[0040] Optionally, the visual detection module includes:

[0041] Image sensor used to acquire images of the wearer's eye area;

[0042] A depth sensor is used to collect the depth value of each pixel in the wearer's eye area;

[0043] Event vision sensor, used to capture event streams of the wearer's eye movements;

[0044] The processing module is used to obtain the output results of the image sensor, the depth sensor and the event vision sensor to calculate the wearer's eye state information.

[0045] In a second aspect, the present invention provides zoom glasses, including the eye-tracking zoom control system based on visual-electrophysiological multimodal as described above.

[0046] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0047] This invention provides an eye-tracking zoom control system and zoom glasses based on visual-electrophysiological multimodal approaches. By constructing a collaborative working mechanism between visual signals and EOG signals, the visual detection module calibrates and constrains the EOG signals, and the EOG detection module compensates for eye-tracking estimation results under visual degradation conditions. This significantly improves the robustness and continuity of the system in complex environments while ensuring spatial accuracy. Attached Figure Description

[0048] Figure 1 This is an external view structural diagram of the intelligent zoom LCD glasses system provided by the present invention;

[0049] Figure 2 This is an internal view structure diagram of the intelligent zoom LCD glasses system provided by the present invention;

[0050] The diagram is marked as follows:

[0051] 1. Variable focus optical element; 2. Modal sensing unit; 201. Visual inspection module; 202. EOG detection module; 203. Ambient light sensing module; 3. Cross-modal adaptive calibration unit; 4. Focusing control unit. Detailed Implementation

[0052] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0053] Example 1

[0054] like Figure 1 and Figure 2 As shown, this embodiment of the invention provides an eye-tracking zoom control system based on visual-electrophysiological multimodal approaches, comprising:

[0055] 1. A variable focus optical element 1 is disposed in the optical imaging path of the eyeglasses and has at least one control area.

[0056] 2. Modal sensing unit 2, including visual detection module 201, EOG detection module 202 and ambient light sensing module 203, are used to collect the wearer's eye state information, periocular potential information and ambient light information when the glasses are worn.

[0057] 3. Cross-modal adaptive calibration unit 3 is used to calibrate the eye state information by means of periocular potential information when the ambient light information does not meet the preset light conditions.

[0058] 4. Focus control unit 4 is used to calculate the gaze direction and gaze distance of the eyeball based on the eyeball state information, determine the target control area based on the gaze direction, generate focus control parameters based on the gaze distance, and drive the target control area to focus through the focus control parameters.

[0059] A. Regarding the variable focus optical element 1, specifically in this embodiment, the variable focus optical element 1 is treated as a whole as a control region or divided into multiple independent control regions, and the boundaries of the control regions are circular or approximately circular continuous curves. In other optional embodiments, those skilled in the art can set the control region to any closed-loop shape as needed. In this embodiment, setting it to a circular or approximately circular continuous curve can reduce liquid crystal orientation distortion, electric field discontinuity, or optical performance fluctuations caused by boundary sharp edges, thereby improving the focusing stability and optical consistency of the liquid crystal lens.

[0060] The embodiments of the present invention provide a choice between a single control region and multiple independent control regions. A single control region only requires one drive channel, which has low hardware and manufacturing costs, but the overall adjustment power consumption is high and lacks specificity. Multiple independent control regions require a corresponding number of drive channels, which has higher hardware and manufacturing costs, but because the area of ​​each control region is reduced, the power consumption is lower, and the focus on the gaze direction is stronger, resulting in better visual effects.

[0061] B. Regarding the visual detection module 201, specifically in this embodiment, it includes an image sensor (RGB), a depth sensor (ToF), an event vision sensor (DVS / EVS), and a processing module. The image sensor is used to acquire images of the wearer's eye region; the depth sensor is used to acquire the depth value of each pixel in the wearer's eye region; and the event vision sensor is used to acquire the event stream of the wearer's eye movements. The processing module is used to obtain the output results of the image sensor, depth sensor, and event vision sensor to calculate the wearer's eye state information.

[0062] Eye status information includes, but is not limited to, the horizontal rotation angle of the left eyeball. Horizontal rotation of the right eyeball Horizontal rotation of the eyeball Vertical angle of eyeball Eyeball radius R, interpupillary distance between left and right eyes b.

[0063] C. Regarding the EOG detection module 202, specifically in this embodiment, it includes multiple electrodes, which are respectively disposed on the left nose pad, right nose pad, left temple, right temple of the glasses, and any earlobe. The electrode on the earlobe is used as a reference electrode. This design can effectively cancel common-mode interference caused by head movement or environment, thereby ensuring from an electrical perspective that the captured "bi-eye angle difference" is pure, which is the physical basis for calculating the accurate distance.

[0064] In a low-cost, minimal configuration, multiple electrodes are placed on the left and right nose pads, any temple, and any earlobe of the glasses.

[0065] Periocular potential information includes, but is not limited to, the potentials of multiple electrodes placed around the eye.

[0066] D. Regarding the ambient light sensing module 203, in this specific embodiment, it is set as a light sensor to obtain the ambient light intensity, thereby determining whether the current environment is extremely dark or has strong light interference.

[0067] E. Regarding the cross-modal adaptive calibration unit 3, specifically in this embodiment:

[0068] E.1. When the ambient light information meets the preset lighting conditions, the periocular potential information is calibrated using eye state information, including:

[0069] When the ambient light information meets the preset lighting conditions, acquire the eye state information and periocular potential information of the wearer when looking at multiple calibration points;

[0070] A bioelectrical model of both eyes was established, and the model parameters of the bioelectrical model were fitted based on the eye state information and periocular potential information of multiple calibration points to complete the calibration of the bioelectrical model.

[0071] E.1.1 Establishing a bioelectrical model of both eyes includes:

[0072] If we consider both eyes as bioelectric dipoles, with the cornea and retina as the positive and negative poles respectively, then the bioelectric model of both eyes is as follows:

[0073]

[0074] In the formula, Let be the potential of the i-th electrode when viewing the j-th calibration point. Let be the distances from the cornea and retina of the right eye to the i-th electrode when fixating on the j-th calibration point. Let be the distances from the cornea and retina of the left eye to the i-th electrode when fixating on the j-th calibration point. These are all model parameters for the i-th electrode.

[0075] E.1.2 The model parameters for fitting the bioelectrical model based on eye state information and periocular potential information from multiple calibration points include:

[0076] Distance is calculated based on eye state information from multiple calibration points. :

[0077] By setting any one electrode as the reference electrode, the potential difference between each electrode and the reference electrode is calculated based on the periocular potential information. And based on potential difference Construct the observation vector y; where, The potential of the reference electrode when viewing the j-th calibration point, Let be the potential difference between the i-th electrode and the reference electrode when viewing the j-th calibration point;

[0078] Design matrix X to represent different potential differences. As rows of matrix X, with row number (I-1)*J, the potential difference is... Corresponding model parameters coefficients of model parameters As columns, the number of columns is 4I; I and J are the number of electrodes and the number of calibration points, respectively;

[0079] The coefficients of the model parameters As the element values ​​of matrix X, the model parameters As an unknown matrix The element values, combined with the observation vector y, are used to construct an overdetermined system of equations: ;

[0080] The least squares method is used to solve the overdetermined system of equations. Solve the problem based on the unknown matrix of the solution. Reconstructed potential difference Before calculating the potential difference reconstruction ( )back( ) root mean square error :

[0081]

[0082] When the root mean square error When the error is less than the threshold, the calibrated unknown matrix is ​​obtained. Complete the fitting of model parameters for the bioelectric model.

[0083] Taking an electrode count of I=5 as an example, the potential difference between the first electrode and the first calibration point for:

[0084]

[0085] The model parameters and their coefficients corresponding to the first electrode and the first calibration point are shown in Table 1.

[0086] Table 1: Model parameters and coefficients corresponding to the first electrode and the first calibration point

[0087]

[0088] Similarly, the row of the second electrode has non-zero coefficients only in columns 5-8 and 17-20, the row of the third electrode has non-zero coefficients in columns 9-12 and 17-20, and the row of the fourth electrode has non-zero coefficients in columns 13-16 and 17-20.

[0089] Construct an unknown matrix using the model parameters and coefficients for all electrodes except the reference electrode. And matrix X.

[0090] E.1.3 Calculating distance based on eye state information from multiple calibration points include:

[0091] The distance between the cornea and retina and the center of the eyeball is equal to the radius R of the eyeball. A reference three-dimensional coordinate system is constructed with the center positions of the left and right eyeballs as the origin. The coordinates of the right eyeball cornea, right eyeball retina, left eyeball cornea, and left eyeball retina are calculated based on the eyeball state information when looking at the j-th calibration point.

[0092] The center coordinates of the right and left eyeballs They are respectively:

[0093]

[0094]

[0095] Calculate the corneal coordinates of the right eye :

[0096]

[0097] Calculate the retinal coordinates of the right eye :

[0098]

[0099] Calculate the corneal coordinates of the left eye :

[0100]

[0101] Calculate the coordinates of the left eye's retina :

[0102]

[0103] Obtain the spatial coordinates of the i-th electrode and transform them into a reference three-dimensional coordinate system. Combine the coordinates of the right eye cornea, right eye retina, left eye cornea, and left eye retina to calculate the distance. .

[0104] Let the coordinates of the i-th electrode be... Then the distance for:

[0105]

[0106]

[0107]

[0108] E.1.4. Calibration of ocular state information using periocular potential information includes:

[0109] The potential difference between each electrode and the reference electrode is calculated based on periocular potential information. ,in, The potential of the reference electrode when fixating on the current fixation point. Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point; Let be the potential difference between the i-th electrode and the reference electrode when looking at the current fixation point;

[0110] The potential difference of the reference electrode Use the pre-built mapping table as an index to obtain the gaze coordinates of the current gaze point;

[0111] Calculate the fitted value of the eye state information based on the coordinates of the current gaze point;

[0112] Let the gaze coordinates of the current gaze point be... The fitted value is calculated as follows:

[0113]

[0114]

[0115]

[0116]

[0117] In the formula, These are the horizontal rotation angles of the eyeball. Vertical angle of eyeball Horizontal angle of left eyeball Horizontal rotation of the right eyeball The fitted value is denoted by b, where b is the interpupillary distance between the left and right eyes.

[0118] The final value of the eye state information is obtained by weighted summing of the original and fitted values. This weighted summation includes:

[0119]

[0120] In the formula, These are weighting coefficients. These represent the original, fitted, and final values ​​of the eyeball state information, respectively. Weighting coefficients The degree to which ambient lighting information deviates from preset lighting conditions is negatively correlated;

[0121] The mapping table is a mapping relationship between the potential difference of each electrode relative to the reference electrode and the gaze coordinates of the gaze point, constructed based on the visual detection module 201 and the calibrated bioelectric model.

[0122] Specifically, in this embodiment, the mapping relationship can be constructed using models such as Transformer, RNN, SVR, Kalman filter, and MLP.

[0123] In constructing the mapping relationship, preprocessing techniques such as bandpass filtering, power frequency denoising, drift elimination, and signal normalization can be applied to the periocular potential information to provide cleaner data support for subsequent processing. Simultaneously, the periocular potential information and the gaze coordinates of the fixation point are time-aligned to avoid temporal misalignment that could prevent model convergence.

[0124] F. Regarding the focusing control unit 4, specifically in this embodiment:

[0125] F.1 Calculate the gaze direction and gaze distance based on eye state information:

[0126] Based on the horizontal rotation angle and center position coordinates of the eyeball Calculate the eye's fixation distance and determine the eye's fixation direction based on the horizontal and vertical rotation angles of the eye.

[0127] Eye gaze distance for:

[0128]

[0129]

[0130] In the formula, For a moment, These are the horizontal rotation angles of the left and right eyeballs, respectively. The distance between the left and right eyeballs. ...

[0131] This invention determines the distance of the object being viewed by the wearer based on the convergence angle relationship formed between the horizontal turning angles of the left and right eyes and the distance between the centers of the left and right eyeballs. The entire calculation is efficient and accurate, which can effectively improve the efficiency and precision of the entire focusing process.

[0132] F.2. The focus control parameters generated based on the gaze distance include:

[0133] The mapping relationships between gaze distance and equivalent focal length, equivalent focal length and applied voltage signal, and applied voltage signal and focus control parameters are constructed; the focus control parameters corresponding to gaze distance are obtained through layer-by-layer mapping.

[0134] Example 2

[0135] This invention provides a zoom glasses system, including the eye-tracking zoom control system based on visual-electrophysiological multimodal as described in Embodiment 1 above.

[0136] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0137] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0139] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0140] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A visual-electrophysiological multimodal based eye movement tracking zoom control system, characterized in that, The method comprises the following steps: a variable focus optical element is arranged in the optical imaging path of the glasses, and has at least one regulation area; a multi-modal sensing unit, including a visual detection module, an EOG detection module, and an ambient light sensing module, is used to collect the eye state information, the eye potential information, and the ambient light information of the wearer when wearing the glasses; a cross-modal adaptive calibration unit is used to calibrate the eye state information based on the eye potential information when the ambient light information does not meet the preset light condition; a focusing control unit is used to calculate the gaze direction and the gaze distance of the eye based on the eye state information, determine the target regulation area based on the gaze direction, generate the focusing control parameter based on the gaze distance, and drive the target regulation area to focus based on the focusing control parameter; when the ambient light information meets the preset light condition, the eye potential information is calibrated based on the eye state information, including: when the ambient light information meets the preset light condition, the eye state information and the eye potential information of the wearer when gazing at multiple calibration points are obtained; a bioelectric model of the eyes of both eyes is established, and the model parameters of the bioelectric model are fitted based on the eye state information and the eye potential information of the multiple calibration points to complete the calibration of the bioelectric model; The eye state information includes a left eye horizontal direction rotation angle , a right eye horizontal direction rotation angle , an eye horizontal direction rotation angle , an eye vertical direction rotation angle , an eye radius R, and a left and right eye pupil distance b. the eye potential information includes the potentials of multiple electrodes arranged around the eyes; any electrode is set as a reference electrode, and the calibration of the eye state information based on the eye potential information includes: calculating a potential difference of each electrode relative to the reference electrode based on the eye periorbital potential information wherein, is a potential of the reference electrode when gazing at the current gaze point, is a potential difference of the i-th electrode relative to the reference electrode when gazing at the current gaze point; is a potential difference of the i-th electrode relative to the reference electrode when gazing at the current gaze point; determining a potential difference of the reference electrode obtaining a gaze point coordinate of a current gaze point by indexing a pre-constructed mapping table the fitting value of the eye state information is calculated based on the gaze point coordinates of the current gaze point; the original value and the fitting value of the eye state information are weighted and summed to obtain the final value of the eye state information; wherein the mapping table is constructed based on the visual detection module and the calibrated bioelectric model to map the potential difference of each electrode relative to the reference electrode and the gaze point coordinates of the gaze point.

2. The visual-electrophysiological multi-modal based eye tracking zoom control system according to claim 1, wherein, The electrodes are arranged on the left nose pad, the right nose pad, the left temple, the right temple, and any earlobe of the glasses, or on the left nose pad, the right nose pad, any temple, and any earlobe of the glasses.

3. The visual-electrophysiological multimodal based eye tracking zoom control system according to claim 1, wherein, The establishment of the bioelectric model of the eyes of both eyes includes: both eyes are regarded as bioelectric dipoles, and the cornea and the retina of the eye are positive and negative respectively, so the bioelectric model of the eyes of both eyes is: ； wherein, is the potential of the i-th electrode when fixating the j-th calibration point, is the distance from the cornea and retina of the right eye to the i-th electrode when fixating the j-th calibration point, is the distance from the cornea and retina of the left eye to the i-th electrode when fixating the j-th calibration point, are the model parameters of the i-th electrode.

4. The visual-electrophysiological multi-modal based eye tracking zoom control system of claim 3, wherein, the fitting of the model parameters of the bioelectric model based on the eye state information and the eye potential information of the multiple calibration points includes: Calculating distance based on eye state information of multiple calibration points ; setting any one electrode as a reference electrode, calculating potential difference of each electrode relative to the reference electrode based on eye periorbital potential information , and calculating the gaze direction based on the potential difference constructing an observation vector y; wherein, is the potential of the reference electrode when gazing at the jth calibration point, is the potential difference of the ith electrode relative to the reference electrode when gazing at the jth calibration point; Design a matrix X, which is a matrix of different potential differences As rows of the matrix X, the number of rows is (I-1)*J, and the potential differences Corresponding model parameters And the coefficients of the model parameters As columns, the number of columns is 4I; I and J are the number of electrodes and the number of calibration points, respectively; The coefficients of the model parameters are taken as element values of a matrix X The model parameters are taken as element values of an unknown matrix are combined with the observation vector y to construct an over-determined equation system: Solving the over-determined equation group by using a least square method , reconstructing a potential difference matrix based on a solution result , calculating root mean square errors before and after reconstruction of the potential difference , obtaining a calibrated unknown matrix when the root mean square errors are less than an error threshold , and completing fitting of model parameters of the bioelectric model.

5. The visual-electrophysiological multi-modal based eye tracking zoom control system according to claim 4, wherein, The distance is calculated based on the eye state information of the plurality of calibration points Comprising: the distances from the cornea and the retina to the center of the eye are equal to the eye radius R, a reference three-dimensional coordinate system is constructed with the center positions of the eyes of both eyes as the coordinate origin, and the right eye cornea coordinates, the right eye retina coordinates, the left eye cornea coordinates, and the left eye retina coordinates are calculated based on the eye state information when gazing at the jth calibration point; acquire spatial coordinates of the i-th electrode and convert into the reference three-dimensional coordinate system, combine the right eye eyeball corneal coordinates, the right eye eyeball retina coordinates, the left eye eyeball corneal coordinates and the left eye eyeball retina coordinates to calculate the distance .

6. The visual-electrophysiological multimodal based eye tracking zoom control system of claim 1, wherein, the weighted summation of the original value and the fitting value of the eye state information includes: ； In the formula, is a weighting coefficient, are respectively the original value, the fitting value and the final value of the eyeball state information; is a weighting coefficient The degree of deviation of the ambient light information from the preset light condition is negatively correlated.

7. The visual-electrophysiological multimodal based eye tracking zoom control system of claim 1, wherein, the visual detection module includes: an image sensor for collecting the eye region image of the wearer; a depth sensor for collecting the depth value of each pixel point of the eye region of the wearer; an event visual sensor configured to capture an event stream of the wearer's eye movement; a processing module configured to obtain output results of the image sensor, the depth sensor, and the event visual sensor to calculate eye state information of the wearer.

8. A variable focus spectacle lens characterized by, An eye tracking zoom control system based on visual-electrophysiological multi-modal, comprising the system as claimed in any one of claims 1-7.

Images