Eye movement tracking identification method based on deep learning

Abstract

The invention relates to the field of eye movement tracking identification, in particular to an eye movement tracking identification method based on deep learning. The method comprises the steps that an eye image sequence is captured and preprocessed, and a standardized eye image sequence is generated; performing feature extraction on the standardized eye image sequence to generate an eye feature vector sequence; based on the eye feature vector sequence, using a dynamic coupling stability index algorithm to generate eye movement track features; based on the eye feature vector sequence, non-blink frames are screened through a lightweight convolutional neural network, and based on eye movement track features of the non-blink frames, stable staring frames are screened; and generating a predicted screen gaze point coordinate based on the eye feature vector of the stationary gaze frame. The problem of inaccurate extraction of pupil center, cornea reflection point and eyelid features under different illumination conditions or complex backgrounds is solved; the stability characteristic of the eye movement track is difficult to accurately capture, so that the classification performance is poor; and misjudgment of a blink frame interferes with eye movement trajectory analysis and staring point prediction.

Description

Technical Field

[0001] The present invention relates to the field of eye tracking and recognition, and in particular to an eye tracking and recognition method based on deep learning. Background Art

[0002] Eye tracking technology is a technique that analyzes human visual behavior by detecting and recording eye movements. With the continuous development of fields such as human-computer interaction, psychology, and medical research, eye tracking technology has gradually become an important tool for studying human cognition and visual behavior. Although traditional eye tracking methods can achieve eye movement monitoring to a certain extent, in dynamic environments, especially under complex lighting conditions or when the eye movement state is unstable, the robustness and accuracy of traditional eye tracking methods often cannot meet the needs of practical applications. With the development of deep learning technology, methods based on deep neural networks have gradually become an important research direction in the field of eye tracking. Deep learning can effectively overcome some of the shortcomings of traditional methods by automatically learning features from large amounts of data. In particular, it has shown significant advantages in robustness and accuracy in complex scenarios. The successful application of deep learning in image recognition, object detection, speech recognition and other fields has provided strong support for the advancement of eye tracking technology.

[0003] The above-mentioned eye tracking and recognition methods still have problems such as insufficient eye tracking accuracy, neglect of the impact of blinking interference on eye tracking, and insufficient robustness of dynamic eye movement classification. Summary of the Invention

[0004] The present invention provides an eye movement tracking and recognition method based on deep learning to solve the technical problems that, in the processing of eye images under different lighting conditions or complex backgrounds, uneven lighting or background noise interference may easily lead to inaccurate extraction of pupil center, corneal reflection point and eyelid features; when distinguishing between rapid eye movements and steady gaze, there is a lack of effective dynamic feature analysis methods, which makes it difficult to accurately capture the stability characteristics of eye movement trajectories, resulting in poor classification performance; when detecting blinking status, blink frames are easily misjudged as valid eye movement data, interfering with eye movement trajectory analysis and gaze point prediction.

[0005] The eye tracking and recognition method based on deep learning of the present invention specifically includes the following technical solutions:

[0006] The eye tracking recognition method based on deep learning includes the following steps: S1. Capture an eye image sequence and perform preprocessing operations to generate a standardized eye image sequence; perform feature extraction on the standardized eye image sequence through a pre-trained convolutional neural network to generate an eye feature vector sequence; S2, based on the eye feature vector sequence, use the dynamic coupling stability index algorithm to generate eye movement trajectory features; S3. Based on the eye feature vector sequence, a lightweight convolutional neural network is used to filter non-blink frames and obtain the eye movement trajectory features of the non-blink frames; based on the eye movement trajectory features of the non-blink frames, stable gaze frames are filtered; based on the eye feature vectors of the stable gaze frames, the predicted screen gaze point coordinates are generated.

[0007] Preferably, the S1 specifically includes: The specific implementation process of the preprocessing operation is as follows: performing grayscale conversion on the eye image to obtain a grayscale image; applying Gaussian filtering to smooth the grayscale image to obtain a smoothed eye image; based on the smoothed eye image, locating the eye area using a facial key point detection algorithm, and cropping the smoothed eye image to obtain a cropped eye image; normalizing the cropped eye image to obtain a standardized eye image.

[0008] Preferably, the S1 specifically includes: The pre-trained convolutional neural network extracts spatial features from the standardized eye image through convolution and pooling operations, and outputs an eye feature vector sequence; the eye feature vector sequence includes pupil center coordinates, corneal reflection point coordinates and eyelid feature vectors.

[0009] Preferably, the S2 specifically includes: The dynamic coupling stability index algorithm generates eye movement trajectory features by analyzing the vector change between the pupil center and the corneal reflection point and combining it with the eyelid feature vector.

[0010] Preferably, the S2 specifically includes: In the implementation process of the dynamic coupling stability index algorithm, the pupil-cornea vector is obtained based on the pupil center coordinates and corneal reflection point coordinates in the eye feature vector, and the angle of the pupil-cornea vector and the modulus of the pupil-cornea vector are calculated; the pupil-cornea vector of the current frame is compared with the previous frame, the angle change and the modulus change are calculated, and normalization is performed to obtain the normalized angle change and the normalized modulus change.

[0011] Preferably, the S2 specifically includes: An exponential decay function is applied to the normalized angle change and the normalized modulus change, and hyperparameters are introduced to quantify the dynamic stability of eye movements.

[0012] Preferably, the S2 specifically includes: By calculating the cosine similarity between the eyelid feature vectors of the current frame and the previous frame and introducing hyperparameters, a linear weighted term is obtained.

[0013] Preferably, the S2 specifically includes: The output of the exponential decay function and the linear weighting term are combined to generate eye movement trajectory features.

[0014] Preferably, the S3 specifically includes: The eyelid feature vector and pupil center coordinates are concatenated and input into a lightweight convolutional neural network to output a binary label. When the lightweight convolutional neural network output is 0, it indicates that the current frame is a non-blink frame.

[0015] Preferably, the S3 specifically includes: The eye movement trajectory features of non-blink frames are obtained and input into the support vector machine classifier to distinguish rapid eye movements from steady gaze and filter out steady gaze frames; based on the pupil center coordinates and corneal reflection point coordinates in the eye feature vector of the steady gaze frame, the relative coordinates of the pupil center and the corneal reflection point are calculated, and the relative coordinates of the pupil center and the corneal reflection point are input into the multi-layer perceptron model to generate the predicted screen gaze point coordinates.

[0016] The beneficial effects of the technical solution of the present invention are: 1. The dynamic coupling stability index algorithm comprehensively analyzes the vector change (including angle and modulus) between the pupil center and the corneal reflection point, as well as the eyelid feature vector, to generate a characteristic value that represents the dynamic stability of the eye movement trajectory. The inverse tangent function is used to calculate the angular change of the pupil-corneal vector to quantify the dynamic nature of the eye movement direction, and the modulus change is used to capture the spatial displacement of the eyeball, compensating for the limitations of a single angle change. The cosine similarity of the eyelid feature vector is used to incorporate the continuity information of eyelid opening and closing to enhance the robustness of the eye movement trajectory features. Multi-dimensional feature coupling analysis enables accurate distinction between rapid eye movements and steady gaze, providing high-quality feature input for subsequent eye movement classification and gaze point prediction, significantly improving the accuracy and reliability of eye movement trajectory recognition.

[0017] 2. We introduce normalized angular variation and normalized modulus variation, combined with an exponential decay function, to quantify the dynamic stability of eye movements and ensure the robustness of eye trajectory features to lighting changes, noise interference, and small head movements. The exponential decay function can correspond to lower stability values ​​for rapid eye movements and higher stability values ​​for steady gaze, thereby effectively distinguishing different eye movement states. At the same time, by linearly weighting the cosine similarity of the eyelid feature vectors, we further enhance the adaptability of the eye trajectory features to blinking or eyelid shape changes. This allows us to maintain the stability of the eye trajectory features in a variety of real-world scenarios (such as different lighting conditions or changes in the user's head posture), thereby improving the practicality of eye tracking and recognition.

[0018] 3. By introducing a time window, the eye movement trajectory features are smoothed, reducing the impact of short-term noise while retaining the dynamic characteristics of eye movements. Hyperparameters are introduced to control the contribution of angle change, modulus change, and eyelid similarity to stability, allowing flexible adjustment of hyperparameters according to specific application scenarios to balance real-time performance and stability. This allows the eye movement trajectory features to quickly respond to eye movement changes while maintaining a smooth trajectory representation, making them suitable for application scenarios such as real-time human-computer interaction, eye tracking, and gaze point prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of the eye tracking and recognition method based on deep learning described in the present invention. DETAILED DESCRIPTION

[0020] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0021] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0022] The specific scheme of the eye tracking and recognition method based on deep learning provided by the present invention is described in detail below with reference to the accompanying drawings.

[0023] Refer to the attached Figure 1 , which shows a flow chart of an eye tracking and recognition method based on deep learning provided by an embodiment of the present invention, the method includes the following steps: S1. Capture an eye image sequence and perform preprocessing operations to generate a standardized eye image sequence; perform feature extraction on the standardized eye image sequence through a pre-trained convolutional neural network to generate an eye feature vector sequence; Use an infrared camera to capture eye image sequences at a fixed frame rate (e.g., 60 frames per second), and perform preprocessing operations on the eye image sequences to generate standardized eye image sequences. ; The specific implementation process of the preprocessing operation is as follows: by performing grayscale conversion on the eye image, the RGB image is converted into a grayscale image to reduce the interference of color information and highlight the eye features; the grayscale image is smoothed by applying Gaussian filtering to obtain a smoothed eye image to eliminate noise interference and ensure that the eye image is smooth and the features are clear; based on the smoothed eye image, the eye area is located by a facial key point detection algorithm (such as Dlib or MediaPipe), and the smoothed eye image is cropped to a fixed size (such as 128×128 pixels), retaining only the eye area to reduce the influence of irrelevant background information, thereby obtaining a cropped eye image; the pixel values ​​of the cropped eye image are normalized to the range of 0 to 1 to ensure consistency in subsequent processing, thereby obtaining a standardized eye image; the grayscale conversion adopts a standard RGB to grayscale weighted formula derived from the theory of human visual perception to ensure that the pupil and corneal reflection point are highlighted; the Gaussian filter is based on the smoothing technology in image processing and is used for noise suppression; the methods used in the above preprocessing process are technical means well known to those skilled in the art and are not described in detail here; Through grayscale conversion, noise filtering, image cropping and normalization, the effects of factors such as lighting changes and background interference are eliminated, ensuring that the eye image quality meets the input requirements of the deep learning model and provides a stable foundation for feature extraction; A pre-trained convolutional neural network (CNN) is used to extract key features from a standardized eye image sequence and generate an eye feature vector sequence containing pupil center coordinates, corneal reflection point coordinates, and eyelid feature vectors. ,in Indicates The pupil center coordinates at time , Indicates The coordinates of the corneal reflection point at the moment, Indicates The eyelid feature vector at the moment, such as the opening and closing angle, is implemented as follows: The standardized eye image is input into a pre-trained convolutional neural network (CNN) (such as ResNet or MobileNet). The pre-trained convolutional neural network (CNN) automatically extracts spatial features from the standardized eye image through multi-layer convolution and pooling operations. For the pupil center and corneal reflection point, the pre-trained convolutional neural network (CNN) predicts two-dimensional coordinates through the regression branch, representing the pixel positions of the pupil center and corneal reflection point in the standardized eye image, respectively, and extracts the eyelid feature vector through the convolution layer to generate a high-dimensional vector (typically 64 dimensions) for characterizing the geometric features of the eyelid contour, such as the opening and closing angle and the normalized representation of the key points of the upper and lower eyelids; the final output is an eye feature vector sequence containing the pupil center coordinates, corneal reflection point coordinates and eyelid feature vectors; the above-mentioned convolutional neural network is a technical means well known to those skilled in the art and will not be described in detail here.

[0024] S2, based on the eye feature vector sequence, use the dynamic coupling stability index algorithm to generate eye movement trajectory features; Based on the eye feature vector sequence, the dynamic coupling stability index algorithm is used to analyze the vector changes (including angle and modulus) between the pupil center and the corneal reflection point. Combined with the eyelid feature vector, the eye movement trajectory features are generated to characterize the dynamic stability of the eye movement trajectory and to distinguish between rapid eye movement and steady gaze. The specific implementation process is as follows: Extract the pupil center coordinates and corneal reflection point coordinates of the current frame from the eye feature vector, calculate the difference between the pupil center coordinates and the corneal reflection point coordinates, and obtain the pupil-corneal vector, which represents the relative position of the eyeball relative to the corneal reflection point and can reflect the rotation direction and amplitude of the eyeball; calculate the angle of the pupil-corneal vector through the inverse tangent function to characterize the direction of eye movement; calculate the modulus of the pupil-corneal vector to represent the distance between the pupil center and the corneal reflection point, and reflect the change in eye movement position; compare the pupil-corneal vector of the current frame with that of the previous frame, and calculate the angle change (direction change) and modulus change (position change); to eliminate the dimension effect, divide the angle change by the mean of the angle change within the time window to obtain the normalized angle change, and divide the modulus change by the mean of the modulus change within the time window to obtain the normalized modulus change; An exponential decay function is applied to the normalized angular change and the normalized modulus change. The angular change of the pupil-corneal vector (characterizing the direction of eye movement) and the modulus change (characterizing the amplitude of eye movement) are coupled together through the exponential decay function to quantify the dynamic stability of eye movement. This ensures that larger angular or modulus changes correspond to lower stability values ​​to reflect the dynamic characteristics of rapid eye movement, while smaller angular or modulus changes correspond to higher stability values ​​to reflect the characteristics of steady gaze. Two hyperparameters are introduced to control the contribution of angle change and modulus change to stability, respectively. The hyperparameters are adjusted according to the actual scenario. The cosine similarity between the eyelid feature vectors of the current frame and the previous frame is further calculated to characterize the continuity of the eyelid shape (such as the stability of the eyelid opening and closing or the contour). Hyperparameters are introduced to obtain a linear weighted term to control the contribution of the eyelid feature to the eye movement trajectory feature, thereby enhancing the robustness of the eye movement trajectory feature. Based on pupil-corneal reflection technology, time series analysis, exponential decay model and cosine similarity, the calculation formula of eye movement trajectory features is constructed: , in, Indicates The eye movement trajectory characteristics at each moment are used to reflect the stability of eye movement; Indicates the time window (length is The average operation of the frame is used to smooth the eye movement trajectory characteristics to capture short-term eye movement dynamics and reduce the influence of noise; Indicates the time window size, which is used to control the time range of eye movement trajectory characteristics to balance real-time and stability. It is set according to expert experience and the recommended value is ; represents an exponential decay term, which is used to characterize the dynamic changes of the pupil-cornea vector. It quantifies eye movement stability by applying exponential decay to the normalized angle change and the normalized modulus change. During rapid eye movement, the normalized angle change and the normalized modulus change are large, and the exponential term tends to 0, indicating low eye movement stability. During steady gaze, the normalized angle change and the normalized modulus change are small, and the exponential term tends to 1, indicating high eye movement stability. Represents the hyperparameter that controls the weight of angle change, which is used to adjust the contribution of angle change to exponential decay to balance dynamics and stability. It is set according to the specific implementation scenario and has a value range of ; Indicates the The angle of the pupil-cornea vector represents the direction of eye rotation based on the relative position of the pupil center and the corneal reflection point. The calculation formula is: , represents the inverse tangent function, Indicates in The pupil center coordinates of the frame, Indicates in The coordinates of the corneal reflection point of the frame; Indicates the Frame (current frame) and The angle change of the frame (previous frame) is used to reflect the dynamic changes in the eye movement direction. The angle change is large during rapid eye movement. Represents the mean of the angle change within the time window, which is used to normalize the angle change; Indicates the The pupil-cornea vector of the frame represents the displacement of the pupil center relative to the corneal reflection point. It is used to characterize the amplitude and direction of eye rotation. The calculation formula is: ; Indicates the Frame (current frame) and The change in vector modulus of the frame (previous frame) is used to reflect the dynamic change of eye position to solve the problem of rapid eye movement despite consistent angles; Indicates the mean value of the modulus length change within the time window, which is used to normalize the modulus length change; It represents the hyperparameter that controls the weight of modulus change, and is used to adjust the contribution of modulus change to exponential decay to balance dynamics and stability. It is set according to the specific implementation scenario and has a value range of ; represents a linear weighted term based on the cosine similarity of the eyelid feature vectors to enhance the robustness of the eye movement trajectory features. The cosine similarity is low during rapid eye movement or blinking, and high during steady gaze. Represents the hyperparameter that controls the cosine similarity weight, which is used to adjust the contribution of eyelid shape continuity to eye movement trajectory characteristics to enhance robustness. It is set according to the specific implementation scenario and the value range is ; Represents the cosine similarity of the eyelid feature vector, which is used to characterize the Frame (current frame) and The continuity of the eyelid shape in the previous frame (such as the opening and closing angle or contour change), the value range is The calculation of mean and cosine similarity in the above formula are both well known to those skilled in the art and will not be described in detail here. Angle changes reflect the dynamics of eye movement direction. During rapid eye movement (REM), the eyeball turns rapidly, resulting in larger angle changes. During steady gaze, the eyeball hardly moves, resulting in smaller angle changes. Modulus changes reflect changes in the distance between the pupil center and the corneal reflection point, capturing the eyeball's tiny displacement in space to compensate for the inability to effectively distinguish REM based solely on angle changes. The cosine similarity of eyelid feature vectors is used to characterize the continuity of eyelid opening and closing or contour (such as changes in key point distances when closed). High similarity indicates stable eyelid shape, while low similarity may reflect shape changes during blinking or REM. The linear weighting term can improve the robustness of eye trajectory features to lighting, noise, and tiny head movements, ensuring the reliability of eye trajectory features in different scenarios. S3. Based on the eye feature vector sequence, a lightweight convolutional neural network is used to filter non-blink frames and obtain the eye movement trajectory features of the non-blink frames. Based on the eye movement trajectory features of the non-blink frames, stable gaze frames are filtered. Based on the eye feature vectors of the stable gaze frames, the predicted screen gaze point coordinates are generated. The eyelid feature vector and pupil center coordinates are concatenated and input into a lightweight convolutional neural network (BlinkNet) to filter non-blink frames. The eyelid feature vector is used to capture the geometric shape and dynamic changes of the eyelid, and the pupil center coordinates assist in determining pupil visibility. The lightweight convolutional neural network (BlinkNet) includes convolutional layers and fully connected layers, introduces a Sigmoid activation function, and outputs a binary label indicating whether the current frame is a blink state (1 for blinking, 0 for non-blinking). The lightweight convolutional neural network (BlinkNet) learns the patterns of eyelid dynamics and pupil position through training data (including labeled blinking and non-blinking frames) to ensure high-precision detection. This is a technical means well known to those skilled in the art and will not be described in detail here. For non-blink frames (i.e., when the output of the lightweight convolutional neural network (BlinkNet) is 0), eye movement trajectory features are obtained and input into a support vector machine (SVM) classifier to distinguish between rapid eye movements and steady gaze. The support vector machine (SVM) classifier uses a linear kernel or a radial basis function kernel (RBF kernel) to learn the separating hyperplane between rapid eye movements and steady gaze through training data. The eye movement trajectory feature values ​​corresponding to rapid eye movements are lower (indicating larger angle or modulus changes), and the eye movement trajectory feature values ​​corresponding to steady gaze are higher (indicating smaller dynamic changes). The SVM outputs a classification label (0 for rapid eye movement, 1 for steady gaze). BlinkNet combines eyelid dynamics and pupil position to accurately detect blinks. Based on eye trajectory features, it uses Support Vector Machine (SVM) to robustly classify rapid eye movements and steady gazes to ensure high overall classification accuracy. Based on the pupil center coordinates and corneal reflection point coordinates in the eye feature vector of the filtered stable gaze frame (classification label is 1), the relative coordinates of the pupil center and the corneal reflection point are calculated, and the relative coordinates of the pupil center and the corneal reflection point are input into a multi-layer perceptron (MLP) model to generate the predicted screen gaze point coordinates. The multi-layer perceptron (MLP) model is a technical means well known to those skilled in the art and will not be described in detail here.

[0025] In summary, the eye tracking and recognition method based on deep learning is completed.

[0026] The order in which the embodiments of the invention are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0027] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0028] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. Eye tracking and recognition method based on deep learning, characterized by: The following steps are involved: S1. Capture an eye image sequence and perform preprocessing operations to generate a standardized eye image sequence; perform feature extraction on the standardized eye image sequence through a pre-trained convolutional neural network to generate an eye feature vector sequence; S2, based on the eye feature vector sequence, use the dynamic coupling stability index algorithm to generate eye movement trajectory features; S3. Based on the eye feature vector sequence, a lightweight convolutional neural network is used to filter non-blink frames and obtain the eye movement trajectory features of the non-blink frames; Based on the eye movement trajectory features of non-blink frames, stable gaze frames are screened; based on the eye feature vectors of stable gaze frames, the predicted screen gaze point coordinates are generated.

2. The eye tracking and recognition method based on deep learning according to claim 1, characterized in that: Said S1 specifically includes: The specific implementation process of the preprocessing operation is as follows: performing grayscale conversion on the eye image to obtain a grayscale image; applying Gaussian filtering to smooth the grayscale image to obtain a smoothed eye image; based on the smoothed eye image, locating the eye area through a facial key point detection algorithm, and cropping the smoothed eye image to obtain a cropped eye image; normalizing the cropped eye image to obtain a standardized eye image.

3. The eye tracking and recognition method based on deep learning according to claim 2, characterized in that: Said S1 specifically includes: The pre-trained convolutional neural network extracts spatial features from the standardized eye image through convolution and pooling operations, and outputs an eye feature vector sequence; the eye feature vector sequence includes pupil center coordinates, corneal reflection point coordinates and eyelid feature vectors.

4. The eye tracking and recognition method based on deep learning according to claim 3, characterized in that: Said S2 specifically includes: The dynamic coupling stability index algorithm generates eye movement trajectory features by analyzing the vector change between the pupil center and the corneal reflection point and combining it with the eyelid feature vector.

5. The eye tracking and recognition method based on deep learning according to claim 4, characterized in that: Said S2 specifically includes: In the implementation process of the dynamic coupling stability index algorithm, the pupil-cornea vector is obtained based on the pupil center coordinates and corneal reflection point coordinates in the eye feature vector, and the angle of the pupil-cornea vector and the modulus of the pupil-cornea vector are calculated; the pupil-cornea vector of the current frame is compared with the previous frame, the angle change and the modulus change are calculated, and normalization is performed to obtain the normalized angle change and the normalized modulus change.

6. The eye tracking and recognition method based on deep learning according to claim 5, characterized in that: Said S2 specifically includes: An exponential decay function is applied to the normalized angle change and the normalized modulus change, and hyperparameters are introduced to quantify the dynamic stability of eye movements.

7. The eye tracking and recognition method based on deep learning according to claim 6, characterized in that: Said S2 specifically includes: By calculating the cosine similarity between the eyelid feature vectors of the current frame and the previous frame and introducing hyperparameters, a linear weighted term is obtained.

8. The eye tracking and recognition method based on deep learning according to claim 7, characterized in that: Said S2 specifically includes: The output of the exponential decay function and the linear weighting term are combined to generate eye movement trajectory features.

9. The eye tracking and recognition method based on deep learning according to claim 8, characterized in that: Said S3 specifically includes: The eyelid feature vector and pupil center coordinates are concatenated and input into a lightweight convolutional neural network to output a binary label. When the lightweight convolutional neural network output is 0, it indicates that the current frame is a non-blink frame.

10. The eye tracking and recognition method based on deep learning according to claim 9, characterized in that: Said S3 specifically includes: The eye movement trajectory features of non-blink frames are obtained and input into the support vector machine classifier to distinguish rapid eye movements from steady gaze and filter out steady gaze frames; based on the pupil center coordinates and corneal reflection point coordinates in the eye feature vector of the steady gaze frame, the relative coordinates of the pupil center and the corneal reflection point are calculated, and the relative coordinates of the pupil center and the corneal reflection point are input into the multi-layer perceptron model to generate the predicted screen gaze point coordinates.

Images