A non-contact gaze estimation method based on deep learning models

Abstract

This invention discloses a non-contact gaze estimation method based on a deep learning model, belonging to the field of computer vision technology. The method constructs a multimodal acquisition system using a camera and a controllable near-infrared light source to simultaneously acquire user facial images, depth information, and screen gaze coordinates. A two-stream deep learning network is used to extract and fuse eye surface texture features and head geometric features. The fused features are input into a differentiable eyeball physical model, and physical regularization is performed using personalized physiological parameters to decouple head movement from eye rotation. An adversarial training mechanism is used to learn feature representations that are invariant to interference factors such as eyeglass reflection and lighting changes. Finally, a regression network outputs accurate screen gaze coordinates. This invention achieves high-precision and robust gaze estimation on medium-cost hardware through the synergistic effect of multimodal fusion, physical prior constraints, and adversarial learning, effectively overcoming the influence of individual differences and environmental changes.

Description

Technical Field

[0001] This invention discloses a non-contact gaze estimation method based on a deep learning model, which relates to the field of computer vision technology. Background Technology

[0002] In the development of modern human-computer interaction and intelligent computing, the demand for non-contact gaze estimation technology is growing. With the rapid popularization of technologies such as virtual reality, augmented reality, and autonomous driving, natural interaction between users and devices has become particularly important. As one of the key technologies for achieving intuitive human-computer interaction, gaze estimation, with its non-invasiveness, naturalness, and rich information content, has become an important means for intelligent systems to understand user intentions.

[0003] Traditional gaze estimation methods are primarily based on contact sensors or single vision sensors. Contact methods involve wearing special glasses or head-mounted devices, using infrared light sources and sensors to directly measure eye movements. Non-contact methods mainly rely on ordinary RGB cameras to capture facial images, estimating gaze through feature extraction and model calculation. However, traditional gaze estimation methods have significant limitations in practical applications: on the one hand, while contact devices offer high accuracy, they require users to wear specialized equipment, resulting in poor comfort and limited usage scenarios; on the other hand, non-contact methods based on a single RGB camera are easily affected by interference factors such as changes in ambient lighting, free head movement of the user, and glare from glasses, leading to a significant decrease in estimation accuracy.

[0004] Specifically, traditional solutions suffer from the following technical problems: In terms of feature extraction, single-modal visual features are difficult to effectively cope with complex environmental changes, especially when lighting conditions change drastically or when users wear glasses, resulting in image quality degradation and unstable feature extraction; in terms of head motion compensation, existing methods often rely on strict head fixation or simple geometric assumptions, which cannot effectively handle the gaze decoupling problem caused by free head movement; in terms of personalized adaptation, traditional methods lack the ability to adapt to individual physiological differences of users, and differences in eye parameters among different users will significantly affect the accuracy of gaze estimation; in terms of model robustness, models trained in a single environment are difficult to generalize to diverse real-world application scenarios.

[0005] Therefore, developing a high-precision, non-contact gaze estimation method that can adapt to complex environmental changes, effectively handle free head movements, and possess personalized adaptation capabilities is of great significance for promoting the development of human-computer interaction technology. This invention aims to overcome the technical bottlenecks of traditional methods and achieve high-precision, highly robust gaze estimation through innovative technologies such as multimodal data fusion, physical model guidance, and adversarial learning. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art, this invention provides a non-contact gaze estimation method based on a deep learning model, which addresses the problems mentioned in the background art through the following approach.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a non-contact gaze estimation method based on a deep learning model, comprising:

[0008] S1. Through a hardware system consisting of an RGB-D camera and a programmable near-infrared light source, a series of data are synchronously collected when the user looks at the stimulation point on the screen. The data includes: RGB image sequence, depth image sequence, screen stimulation point coordinates that are precisely synchronized with each image frame, and image data generated under different near-infrared illumination modes.

[0009] S2. Process the acquired RGB image sequence and depth image sequence to extract the user's head three-dimensional pose information, and locate the eye region based on this. Then, through a two-stream deep learning coding network, extract features from the RGB eye image and depth information respectively, and fuse the features into a unified multimodal feature vector.

[0010] S3. Input the unified multimodal feature vector into a network module containing a differentiable eyeball physical model. Based on the preset initial values ​​of user-personalized eyeball physiological parameters, the multimodal feature vector is transformed and regularized in accordance with the optical geometry of the eyeball, and a gaze direction feature constrained by the physical model is output, representing pure eyeball rotation.

[0011] S4. Input the gaze direction features into a network structure containing an adversarial training mechanism, including: a main task regressor responsible for predicting gaze coordinates based on the gaze direction features, and an adversarial discriminator configured to identify the interference factor domain in the input data; through adversarial training, the two-stream deep learning coding network is forced to learn a gaze feature representation that is invariant to the interference factor domain;

[0012] S5. Based on the gaze direction features purified by the adversarial training mechanism, the user's two-dimensional gaze coordinates on the screen are calculated and output through the final regression layer.

[0013] The technical effects and advantages of this invention are as follows:

[0014] 1. This invention achieves efficient extraction and fusion of high-quality features through a multimodal data acquisition system and a dual-stream deep learning coding network. The system utilizes an RGB-D camera and a programmable near-infrared light source to simultaneously acquire image data including texture, depth, and various lighting conditions. The dual-stream coding network extracts apparent texture features and depth geometric features separately, and employs an attention mechanism for intelligent fusion, significantly improving the richness and discriminative power of feature representation, laying a solid foundation for high-precision gaze estimation.

[0015] 2. This invention achieves strong robustness to head movements and adaptability to individual user differences by introducing a differentiable eyeball physical model and personalized parameter initialization. This physical model embeds prior knowledge conforming to biomechanical constraints into the network, explicitly decoupling head rotation from eyeball rotation. Combined with personalized eyeball physiological parameters obtained through a calibration process, the model can accurately represent pure line-of-sight direction, effectively overcoming interference from free head movements and improving cross-user generalization ability.

[0016] 3. This invention achieves domain-invariant feature learning in response to interference factors such as changes in illumination and eyeglass reflections by constructing an adversarial training mechanism. This mechanism jointly optimizes the regression loss of the main task and the adversarial discrimination loss, and utilizes a gradient inversion layer to force the feature encoding network to discard interference information unrelated to the gaze direction. This results in learning highly pure and robust gaze feature representations, significantly improving the estimation accuracy and stability of the system in complex real-world environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is the overall architecture diagram.

[0019] Figure 2 This is a schematic diagram of a multimodal data acquisition system.

[0020] Figure 3 This is a schematic diagram illustrating the working principle of a differentiable physical model of the eyeball. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0022] See Figure 1 As shown, this invention provides an overall architecture diagram of a non-contact gaze estimation method based on a deep learning model. The diagram illustrates the complete processing flow from data acquisition to gaze coordinate output, including five core steps: S1 multimodal data acquisition, S2 head pose extraction and feature fusion, S3 gaze decoupling from physical model constraints, S4 adversarial training feature cleansing, and S5 gaze coordinate output. These steps are connected by explicit data flows, forming an end-to-end gaze estimation solution.

[0023] S1. Through a hardware system consisting of an RGB-D camera and a programmable near-infrared light source, a series of data are simultaneously collected when the user looks at the stimulation point on the screen. The data includes: RGB image sequence, depth image sequence, screen stimulation point coordinates that are precisely synchronized with each image frame, and image data generated under different near-infrared illumination modes.

[0024] See Figure 2 As shown in the diagram, this invention provides a schematic of a multimodal data acquisition system. The diagram illustrates the system's hardware composition and data acquisition process, including three core components: an RGB-D camera, a programmable near-infrared light source, and screen stimulus points. The system acquires four data types simultaneously: RGB image sequences, depth image sequences, screen stimulus point coordinates, and image data under different lighting conditions. The near-infrared light source cycles through four operating modes, providing rich multi-light condition samples for subsequent feature extraction.

[0025] This embodiment details the complete implementation of the multimodal data acquisition system. The system uses an Intel RealSense D455 RGB-D camera, mounted directly above a 24-inch monitor, with the camera maintaining a working distance of 70cm from the user's eyes. The RGB camera acquires 1280×720 pixel image sequences at 30fps, while a depth sensor simultaneously acquires depth image sequences of the same resolution. The depth measurement range is 0.4-6m, with a depth accuracy of ±1mm at 70cm.

[0026] The screen stimulus moves along a predefined Lissajous curve, and the trajectory is defined by parametric equations:

[0027] ;

[0028] ;

[0029] Where t is time in seconds. For example, when t = 1.0 s, the coordinates of the stimulus point are:

[0030] ;

[0031] ;

[0032] The system uses hardware triggering to ensure millisecond-level synchronization between each image frame and the stimulus point coordinates, with a timestamp accuracy of ±1ms. Each data frame contains a complete multimodal data packet: RGB image, depth image, timestamp, stimulus point coordinates (x, y), and the current near-infrared illumination mode ID.

[0033] The programmable near-infrared light source operates in a time-division multiplexing manner, switching cyclically in a preset order under different illumination modes. Each mode has a different combination of light-emitting units, illumination angle, or intensity, so that the synchronously acquired image data contains discriminative features related to the geometry of the eyeball generated by different illumination conditions.

[0034] This embodiment details the operating modes and parameter configurations of the programmable near-infrared light source. The near-infrared light source consists of 16 LEDs with a wavelength of 850nm, divided into four independent control groups: Group 1 (4 LEDs at the top), Group 2 (4 LEDs at the bottom), Group 3 (4 LEDs on the left), and Group 4 (4 LEDs on the right).

[0035] The light source operates in a strict time-division multiplexing mode, with each mode lasting 83.3ms. The complete cycle T is:

[0036] ;

[0037] The specific work sequence is as follows:

[0038] Mode A (0-83.3ms): Activates LED group 1, drive current 100mA

[0039] Mode B (83.3-166.6ms): Activates group 2 LEDs, drive current 100mA

[0040] Mode C (166.6-250ms): Activates group 3+4 LEDs, drive current 80mA

[0041] Mode D (250-333.2ms): Activates all LEDs, drive current 50mA

[0042] Within each complete cycle, the system can acquire image sequences under four different lighting conditions. For example, at time t=100ms, the system is in mode B, activating the lower LED group; at t=200ms, it is in mode C, activating the left and right LED groups. This periodic switching of lighting modes ensures that the acquired image sequences contain discriminative features closely related to the geometry of the eye, such as differences in shadow distribution, iris texture features, and changes in ocular surface reflection features caused by changes in lighting direction.

[0043] S2. Process the acquired RGB image sequence and depth image sequence to extract the user's head three-dimensional pose information, and locate the eye region based on this. Then, through a two-stream deep learning coding network, extract features from the RGB eye image and depth information respectively, and fuse the features into a unified multimodal feature vector.

[0044] This embodiment details the complete processing flow for head 3D pose information extraction, eye region localization, and dual-stream feature fusion. The input data is the complete multimodal dataset output from step S1, including timestamp-synchronized RGB image sequences, depth image sequences, screen stimulus coordinates, and near-infrared illumination pattern IDs.

[0045] Taking a data frame with a timestamp of t=100ms as an example, the RGB image (1280×720 pixels) and depth image (1280×720 pixels) at that moment are processed first. The MediaPipeFaceMesh model is used to detect 468 facial key points, and 6 stable key points are selected: tip of nose (1, coordinates [643, 285]), left corner of eye (33, [580, 270]), right corner of eye (263, [700, 265]), left corner of mouth (61, [590, 380]), right corner of mouth (291, [685, 375]), and tip of chin (199, [640, 450]).

[0046] The 3D coordinates of key points are obtained from depth images, and the calculation formula is as follows:

[0047] ;

[0048] in Let i be the three-dimensional coordinates of the i-th key point in the camera coordinate system. Let K be the inverse of the camera intrinsic parameter matrix. Let be the pixel coordinates of the i-th keypoint in the image. Let (u(i), v(i)) be the depth value of the depth image at position (u(i), v(i)).

[0049] The camera intrinsic parameter matrix K is known as follows:

[0050] ;

[0051] Then the inverse matrix of K is:

[0052] ;

[0053] For the key point at the tip of the nose (u=643, v=285), the depth value D=0.71 meters at this location is read from the depth image in step 1.

[0054] but Further findings revealed:

[0055] ;

[0056] The final calculation yielded:

[0057] ;

[0058] The face region is detected in the RGB image sequence, and a set of predefined facial key points in the face region are located. Based on the depth image sequence, the three-dimensional spatial coordinates corresponding to the set of predefined facial key points are obtained. According to the three-dimensional spatial coordinates of the set of predefined facial key points and the coordinates of the corresponding key points in a standard three-dimensional face model, the rotation vector and translation vector of the head, i.e. the three-dimensional pose information of the head, are calculated by solving a PnP problem.

[0059] This embodiment details the calculation process of the head's three-dimensional pose information. Based on the three-dimensional coordinates of six key points and the corresponding coordinates of a standard three-dimensional face model, the PnP problem is solved:

[0060] ;

[0061] in The weight coefficient for the i-th key point reflects the detection stability of that key point; The three-dimensional coordinates of the i-th keypoint are calculated from the depth image; R represents the coordinates of the i-th key point in the standard 3D face model; R is a 3×3 rotation matrix describing the rotation of the head relative to the camera; t is a 3×1 translation vector describing the translation of the head relative to the camera.

[0062] The weighting coefficients are set as follows:

[0063] Key points for the tip of the nose and the corner of the eye: 1.0;

[0064] Key point for the corner of the mouth: 0.8;

[0065] Key point for the chin: 0.6;

[0066] Using the Levenberg-Marquardt algorithm for iterative optimization, we obtain:

[0067] Rotation vector ,

[0068] Translation vector .

[0069] The eye region is located based on head posture information.

[0070] The left eye region is defined as a rectangle. ,

[0071] The right eye region is defined as .

[0072] Extract binocular RGB image patches (224×224 pixels) and depth image patches (224×224 pixels) as input to the two-stream network.

[0073] Two-stream deep learning coding networks specifically include:

[0074] An RGB stream coding subnetwork, consisting of a convolutional neural network, takes as input an image patch of the eye region located from an RGB image sequence, and is used to extract apparent texture features related to the pupil, iris and corner of the eye.

[0075] A depth / pose stream coding subnetwork, whose input includes a head 3D pose vector calculated from a sequence of depth images and depth patches of the eyes, is used to extract depth geometric features related to the head spatial location and eye geometry;

[0076] A feature fusion module is used to concatenate or weightedly fuse apparent texture features with deep geometric features to output a unified multimodal feature vector.

[0077] In a two-stream deep learning encoding network, the RGB stream uses the ResNet-18 architecture to process the RGB images of the eyes, outputting a 512-dimensional feature vector. The depth / pose flow processes the head pose vector and eye depth map separately: the head pose branch maps the 6-dimensional pose vector to 128-dimensional features through a fully connected network. The depth map branch processes the eye depth map through a 4-layer convolutional network, outputting 128-dimensional features. .

[0078] In a two-stream deep learning coding network, the feature fusion calculation process is as follows:

[0079] ;

[0080] ;

[0081] in It is a 768×3-dimensional learnable parameter matrix. This is the 512-dimensional feature vector output by the RGB stream encoding subnetwork. This is the 128-dimensional feature vector output by the pose flow coding subnetwork. This is the 128-dimensional feature vector output by the deep stream coding subnetwork. The attention weights for each feature stream.

[0082] The final output is a 768-dimensional fused feature vector. This serves as the input for the subsequent line-of-sight estimation module.

[0083] S3. Input the unified multimodal feature vector into a network module containing a differentiable eyeball physical model. Based on the preset initial values ​​of user-personalized eyeball physiological parameters, the multimodal feature vector is transformed and regularized in accordance with the optical geometry of the eyeball, and a gaze direction feature constrained by the physical model is output, representing pure eyeball rotation.

[0084] This embodiment details the complete processing flow of the differentiable eyeball physical model. The processing strictly uses the 768-dimensional unified multimodal feature vector output from step S2.

[0085] F fused =[0.13, -0.07, 0.06, 0.09, 0.11, -0.04, ...]∈R 768 ; and head 3D pose information.

[0086] Preset initial values ​​for user-personalized ocular physiological parameters are obtained as follows: Before the S1 acquisition step, a user-personalized parameter initialization step is performed, which includes:

[0087] S0.1 Instruct the user to keep their head still and look at the calibration points displayed on the screen in different positions in turn;

[0088] S0.2. When the user is looking at each calibration point, the RGB-D camera simultaneously captures the user's facial image and depth information;

[0089] S0.3. Based on the collected depth information, calculate the three-dimensional position of the user's head in the camera coordinate system;

[0090] S0.4. Based on the three-dimensional position of the head, the screen coordinates of multiple calibration points of the user's gaze, and the geometric constraints based on the standard eyeball model, the user's personalized initial values ​​of eyeball physiological parameters are estimated through an optimization algorithm. The initial values ​​of the parameters include the radius of the eyeball and the initial three-dimensional coordinates of the corneal curvature center in the head coordinate system.

[0091] This embodiment details the calculation process of initial values ​​for personalized eye physiological parameters. Before formal data acquisition, the user sequentially focuses on nine calibration points on the screen, including the center point (960, 540) and eight edge points. While the user focuses on the center calibration, the system acquires facial images and depth information to calculate the head position. calib =[0, 0, 0, 0, 0, 0.70]. Based on the data from all 9 calibration points, the objective function is solved using an optimization algorithm:

[0092] ;

[0093] in Let i be the screen coordinates of the i-th calibration point; For eye projection functions; θ represents the position of the eyeball in the head coordinate system; θ represents the personalized parameter to be optimized.

[0094] After optimization using the Levenberg-Marquardt algorithm, initial values ​​of the user's personalized eye physiological parameters were obtained:

[0095] ;

[0096] in The radius of the eyeball, The coordinates of the corneal curvature center are three-dimensional coordinates in the head coordinate system.

[0097] The differentiable eye physics model is a mathematical computation layer embedded in a deep learning network. This model is configured to: receive a unified multimodal feature vector and initial values ​​of the user's personalized eye physiological parameters as input; establish a mapping relationship from the apparent features of the eye to the three-dimensional rotation of the eye based on the optical geometry of the eye, which is achieved through a series of differentiable mathematical operations; the mathematical operations include mapping eye image features to three-dimensional coordinates in the head coordinate system based on personalized eye physiological parameters, and solving for the gaze direction of the eye through vector operations; the output of the model is a feature vector representing the rotation direction of the eye in three-dimensional space. The gradient of the entire computation process can be calculated during backpropagation, enabling joint optimization of the personalized eye physiological parameters and the parameters of the two-stream deep learning coding network.

[0098] See Figure 3As shown, this invention provides a schematic diagram of the working principle of a differentiable eyeball physical model. The diagram illustrates the core processing mechanism of the physical model, including two inputs (a unified multimodal feature vector and initial values ​​of the user's personalized eyeball physiological parameters) and one output (gazing direction features). The model establishes a mapping relationship from the apparent features of the eye to the three-dimensional rotation of the eyeball through differentiable mathematical operations, achieving transformations and regularization that conform to the optical geometry of the eyeball, ensuring that the output gaze direction features accurately represent pure eyeball rotation.

[0099] This embodiment details the computational process of a differentiable eyeball physical model. The model first maps 768-dimensional feature vectors to 3-dimensional unit vectors using a multilayer perceptron:

[0100] ;

[0101] in The network structure is [768, 384, 192, 3], and all operations are kept differentiable.

[0102] Next, the three-dimensional position of the pupil in the head coordinate system is calculated:

[0103] ;

[0104] Then calculate the line-of-sight vector:

[0105] ;

[0106] Finally, vector normalization is performed to output the unit view direction vector:

[0107] ;

[0108] S4. Input the gaze direction features into a network structure containing an adversarial training mechanism, including: a main task regressor responsible for predicting gaze coordinates based on the gaze direction features, and an adversarial discriminator configured to identify the interference factor domain in the input data; through adversarial training, the two-stream deep learning encoding network is forced to learn a gaze feature representation that is invariant to the interference factor domain;

[0109] This embodiment details the complete implementation process of the adversarial training mechanism. The processing strictly utilizes the 3D gaze direction feature output from step S3. dir =[0.384, -0.576, 0.720], and simultaneously combined with the 768-dimensional multimodal feature vector F output from step S2. fused =[0.13, -0.07, 0.06, 0.09, 0.11, -0.04, ...]∈R 768 .

[0110] Adversarial training is achieved by jointly optimizing the regression loss and the classification loss of the discriminator.

[0111] This embodiment illustrates the complete computational framework and optimization process for adversarial training.

[0112] The main task regressor uses a three-layer fully connected network structure. The input is the 3D gaze direction features output by S3, and the output is normalized screen coordinates.

[0113] ,

[0114] ,

[0115] ,

[0116] in The first layer weight matrix maps the 3D input to a 256-dimensional hidden layer; This is the first layer bias vector. The second layer weight matrix maps the 256-dimensional input to a 128-dimensional hidden layer. This is the second layer bias vector; This is the output layer weight matrix; This is the output layer bias vector.

[0117] Based on the above data, the normalized screen coordinates of this embodiment can be obtained as follows: ;

[0118] S5. Based on the gaze direction features purified by the adversarial training mechanism, the user's two-dimensional gaze coordinates on the screen are calculated and output through the final regression layer.

[0119] This embodiment details the complete calculation process for the final line-of-sight coordinate output. The processing strictly utilizes line-of-sight direction features. and normalized screen coordinates .

[0120] The regression layer calculation specifically includes: inputting the gaze direction features purified by the adversarial training mechanism into a regression network composed of fully connected layers; the regression network maps the high-dimensional gaze direction features into a two-dimensional vector through a combination of hierarchical linear transformation and nonlinear activation functions; normalizing the two-dimensional vector to represent normalized coordinates in the screen coordinate system; and finally converting the normalized coordinates into actual screen coordinates in pixels as the final output of gaze estimation.

[0121] This embodiment details the implementation process of the regression layer calculation. To ensure that the output coordinates are within a reasonable range, the regression network output is normalized using Sigmoid.

[0122] ;

[0123] in The processed normalized screen coordinates are This converts the normalized coordinates into actual screen coordinates in pixels. The screen resolution is 1920×1080 pixels.

[0124]

[0125]

[0126] The final actual screen coordinates, converted to pixels, are (1020, 516) pixels.

[0127] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0128] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A non-contact gaze estimation method based on a deep learning model, characterized in that, Includes the following steps: S1. Through a hardware system consisting of an RGB-D camera and a programmable near-infrared light source, a series of data are synchronously collected when the user looks at the stimulation point on the screen. The data includes: RGB image sequence, depth image sequence, screen stimulation point coordinates that are precisely synchronized with each image frame, and image data generated under different near-infrared illumination modes. S2. Process the acquired RGB image sequence and depth image sequence to extract the user's head three-dimensional pose information, and locate the eye region based on this. Then, through a two-stream deep learning coding network, extract features from the RGB eye image and depth information respectively, and fuse the features into a unified multimodal feature vector. The dual-stream deep learning coding network specifically includes: An RGB stream coding subnetwork, consisting of a convolutional neural network, takes as input an image patch of the eye region located from the RGB image sequence, and is used to extract apparent texture features related to the pupil, iris and corner of the eye. A depth / pose stream coding subnetwork, whose input includes a head 3D pose vector calculated from the depth image sequence and depth patches of the eyes, is used to extract depth geometric features related to the head spatial location and eye geometry; A feature fusion module is used to concatenate or weightedly fuse the apparent texture features with the depth geometric features, thereby outputting the unified multimodal feature vector; S3. Input the unified multimodal feature vector into a network module containing a differentiable eyeball physical model. Based on the preset initial values ​​of user-personalized eyeball physiological parameters, the multimodal feature vector is transformed and regularized in accordance with the optical geometry of the eyeball, and a gaze direction feature constrained by the physical model is output, representing pure eyeball rotation. The differentiable eye physics model is a mathematical computation layer embedded in a deep learning network, which is configured to receive the unified multimodal feature vector and the user's personalized initial values ​​of eye physiological parameters as input. Based on the optical geometric properties of the eyeball, a mapping relationship from the apparent features of the eye to the three-dimensional rotation of the eyeball is established. This mapping relationship is realized through a series of differentiable mathematical operations. The mathematical operations include mapping eye image features to three-dimensional coordinates in the head coordinate system based on the personalized eye physiological parameters, and solving for the gaze direction of the eyeball through vector operations; The output of the model is a feature vector representing the rotation direction of the eyeball in three-dimensional space. The gradient of the entire calculation process can be calculated in backpropagation, so that the personalized eyeball physiological parameters and the parameters of the two-stream deep learning coding network can be jointly optimized. S4. Input the gaze direction features into a network structure containing an adversarial training mechanism, including: a main task regressor responsible for predicting gaze coordinates based on the gaze direction features, and an adversarial discriminator configured to identify the interference factor domain in the input data; through adversarial training, the two-stream deep learning coding network is forced to learn a gaze feature representation that is invariant to the interference factor domain; S5. Based on the gaze direction features purified by the adversarial training mechanism, the user's two-dimensional gaze coordinates on the screen are calculated and output through the final regression layer.

2. The non-contact gaze estimation method based on a deep learning model according to claim 1, characterized in that, The programmable near-infrared light source operates in a time-division multiplexing manner, switching cyclically in a preset order under different illumination modes. Each mode has a different combination of light-emitting units, illumination angle, or intensity, so that the synchronously acquired image data contains discriminative features related to the geometry of the eyeball generated by different illumination conditions.

3. The non-contact gaze estimation method based on a deep learning model according to claim 1, characterized in that, The specific steps for extracting the user's three-dimensional head pose information are as follows: S2.1 Detect the face region from the RGB image sequence and locate a set of predefined facial key points in the face region; S2.

2. Based on the depth image sequence, obtain the three-dimensional spatial coordinates corresponding to the set of predefined facial key points; S2.

3. Based on the three-dimensional spatial coordinates of the predefined set of facial key points and the coordinates of the corresponding key points in a standard three-dimensional face model, the rotation vector and translation vector of the head, i.e., the three-dimensional pose information of the head, are calculated by solving a PnP problem.

4. The non-contact gaze estimation method based on a deep learning model according to claim 1, characterized in that, The preset initial values ​​of the user-personalized eye physiological parameters are obtained through the following methods: Prior to the S1 data acquisition step, a user-personalized parameter initialization step is performed, which includes: S0.1 Instruct the user to keep their head still and look at the calibration points displayed on the screen in different positions in turn; S0.

2. When the user is looking at each calibration point, the RGB-D camera synchronously acquires the user's facial image and depth information; S0.

3. Based on the collected depth information, calculate the three-dimensional position of the user's head in the camera coordinate system; S0.

4. Based on the three-dimensional position of the head, the screen coordinates of multiple calibration points gazed at by the user, and the geometric constraints based on the standard eyeball model, the user's personalized initial values ​​of eyeball physiological parameters are estimated through an optimization algorithm. The initial values ​​of the parameters include the radius of the eyeball and the initial three-dimensional coordinates of the corneal curvature center in the head coordinate system.

5. The non-contact gaze estimation method based on a deep learning model according to claim 1, characterized in that, The adversarial training is achieved by jointly optimizing the regression loss and the classification loss of the discriminator.

6. The non-contact gaze estimation method based on a deep learning model according to claim 1, characterized in that, The regression layer calculation specifically includes: The gaze direction features purified by the adversarial training mechanism are input into a regression network consisting of fully connected layers; This regression network maps high-dimensional gaze direction features into a two-dimensional vector through a combination of hierarchical linear transformation and nonlinear activation functions. The two-dimensional vector is normalized to represent normalized coordinates in the screen coordinate system. Finally, the normalized coordinates are converted into actual screen coordinates in pixels, which are used as the final output of the gaze estimation.

Images