An eye movement tracking method and system based on a hybrid attention mechanism

Abstract

The application discloses an eye movement tracking method and system based on a mixed attention mechanism and belongs to the technical field of computer vision and artificial intelligence. By constructing a double-channel input convolutional neural network, a configurable channel attention module and a spatial attention module are integrated to form a mixed attention mechanism, and an angle loss function based on a three-dimensional geometric model is designed, so that the estimation accuracy of a gaze point and visual consistency are significantly improved. The method has the characteristics that the requirements of high-precision gaze estimation and real-time inference speed are met at the same time, the average angle error reaches 2.8 degrees, and the inference speed is kept at 120 frames per second. The insertion position of the attention module is dynamically adjusted through a configuration file, the gaze estimation performance of the model is improved, and then the user experience of application scenarios such as virtual reality, augmented reality and driver monitoring is improved, so that the method has a wide application prospect.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to an eye-tracking method and system based on deep learning and attention mechanisms. Background Technology

[0002] Eye-tracking technology aims to estimate a user's gaze point by analyzing images of the human eye, and is a key technology for achieving natural interaction, behavior analysis, and medical diagnosis. Existing eye-tracking methods are mainly divided into feature-based methods and deep learning-based methods.

[0003] Feature-based methods typically rely on the detection and geometric modeling of eye features such as the pupil center and corneal reflector. These methods are sensitive to image quality, lighting conditions, and head movement, and lack robustness in complex scenes. Deep learning-based methods, through end-to-end training, directly regress gaze coordinates from images, avoiding the limitations of manual feature design. However, existing deep learning methods still suffer from the following problems:

[0004] Insufficient feature utilization: Existing CNN architectures typically process the left and right eye images separately or simply stitch them together, failing to effectively model the collaborative relationship between the two eyes;

[0005] The loss function is poorly designed: pixel-level loss functions such as L1 and L2 are often used, which do not match the geometric characteristics of human vision, resulting in insufficient angle estimation accuracy.

[0006] Poor balance between model complexity and performance: In order to improve accuracy, the network depth and parameter size are often increased, resulting in high consumption of computing resources and difficulty in deployment to edge devices;

[0007] Inefficient training: There is a lack of customized optimization strategies for the gaze estimation task, resulting in slow training convergence and a tendency to get stuck in local optima.

[0008] Therefore, there is a need for an eye-tracking method that can fully exploit binocular coordination features, conform to visual geometry, and balance accuracy and efficiency. Summary of the Invention

[0009] The purpose of this invention is to provide an eye-tracking method and system based on a hybrid attention mechanism to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution: an eye-tracking method based on a hybrid attention mechanism, comprising the following steps:

[0011] S1. Data Preprocessing: The raw eye-tracking data is parsed and aligned with coordinates. Frame identifiers and gaze information are extracted using regular expressions. The verified left and right eye images and gaze coordinates are stored in a compressed HDF5 format according to a session-frame structure, achieving data structuring and efficient storage.

[0012] S2. Construct a deep neural network model with a hybrid attention mechanism. Use dual-channel input to process left and right eye images respectively. Design a basic feature extraction network containing multiple cascaded convolutional blocks. The number of feature map channels increases layer by layer while the spatial size decreases. Integrate channel attention module and spatial attention module to form a hybrid attention mechanism. The channel attention module focuses on important feature channels based on the Squeeze-and-Excitation mechanism. The spatial attention module generates spatial attention weights by calculating the statistical information of the feature map channel dimensions.

[0013] S3. Define a composite loss function based on geometric constraints, which includes pixel-level accuracy loss and three-dimensional geometric angle loss. The pixel-level accuracy loss uses the L1 loss function to calculate the absolute error between the predicted coordinates and the true coordinates. The three-dimensional geometric angle loss converts the two-dimensional screen gaze coordinates into a three-dimensional spatial vector and calculates the angle between the predicted vector and the true vector as the angle error.

[0014] S4. Efficient training and optimization strategies are adopted, including mixed precision training, dynamic learning rate scheduling, and early stopping strategy. Mixed precision training uses different precision calculations in forward and backward propagation to balance memory usage and numerical stability. Dynamic learning rate scheduling adjusts the learning rate during training to improve convergence speed. The early stopping strategy terminates training early based on the angle error of the validation set and saves the best model.

[0015] Preferably, the data preprocessing in S1 specifically includes: The left and right eye images are used as two independent input channels to form an input tensor of shape [2, H, W]; 2 represents the number of input channels, the first channel is the left eye image, and the second channel is the right eye image; H represents the height of a single eye image in pixels; W represents the width of a single eye image in pixels. Normalize the input tensor.

[0016] Preferably, the insertion position of the channel attention module in S2 is dynamically specified through a configuration file, supporting selective insertion after different convolutional layers of the feature extraction network.

[0017] Preferably, the composite loss function in S3 includes: The L1 loss term calculates the absolute error between the predicted coordinates and the true coordinates. The angle loss term optimizes the conversion of two-dimensional screen coordinates into three-dimensional spatial angle errors.

[0018] Preferably, the angle error in S3 is: ; In the above formula, (x1,y1) are the actual gaze coordinates, (x2,y2) are the predicted gaze coordinates, and z is the preset vertical distance parameter from the eyeball to the screen.

[0019] Preferably, the efficient training and optimization strategy in S4 employs at least one of the following: Mixed-precision training, using float16 precision for forward propagation and gradient calculation; and / or Dynamic learning rate scheduling, employing the OneCycleLR strategy; and / or Use the AdamW optimizer to update parameters.

[0020] On the other hand, the present invention also provides a system applicable to the eye-tracking method based on the hybrid attention mechanism as described above, comprising: The data preprocessing module is used to convert raw eye-tracking data into a structured storage format; The data loading module is used to load preprocessed eye-tracking data in batches. An eye-tracking model includes a feature extraction network and a regression output layer, wherein the feature extraction network embeds a configurable channel attention module and a spatial attention module; The training optimization module is used to train the eye-tracking model; The testing and evaluation module is used to evaluate the performance of the trained eye-tracking model.

[0021] Preferably, the data preprocessing module stores the raw data in HDF5 format, with a session-frame hierarchy, and each frame group contains a left-eye image, a right-eye image, and gaze coordinate attributes.

[0022] Preferably, the channel attention module includes: A global average pooling layer is used to generate channel description vectors; Two fully connected layers are used to learn the dependencies between channels; The sigmoid activation function is used to generate channel attention weights.

[0023] Preferably, the spatial attention module includes: a channel-dimensional average value calculation unit; a channel-dimensional maximum value calculation unit; a stitching layer for stitching the average value map and the maximum value map along the channel dimension; a convolutional layer for generating a spatial attention weight map; and a sigmoid activation function for generating spatial attention weights.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) This method effectively solves the problems of insufficient feature extraction and insufficient modeling of binocular coordination relationship in traditional convolutional neural networks. Through the collaborative work of configurable channel attention and spatial attention modules, it has the characteristics of simple structure and easy integration and implementation. It can significantly improve the gaze estimation accuracy and constrain the angle error to be less than the preset threshold.

[0026] (2) Compared with traditional deep learning methods, only configurable attention modules and geometric constraint loss functions are added. The function of hybrid attention mechanism can be realized without changing the basic network architecture. It can be widely used in various eye-tracking scenarios such as virtual reality, augmented reality, and driver monitoring. At the same time, the model parameter tuning process is very convenient and supports dynamic adjustment of the attention module insertion position through configuration files.

[0027] (3) In the design of the loss function, the present invention introduces the angle error calculation based on the three-dimensional geometric model, which can further improve the perception consistency of the controller; the dynamically adjustable depth parameter z and the configurable attention module insertion strategy also enhance the system's adaptability, enabling it to adapt to different eye-tracking scenarios and application requirements, and making its application range wider. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 The flowchart of the eye-tracking method based on the hybrid attention mechanism provided by the present invention is shown.

[0030] Figure 2 This is a schematic diagram of the hybrid attention neural network structure of the present invention.

[0031] Figure 3 This is a detailed structural diagram of the channel attention module and the spatial attention module of the present invention.

[0032] Figure 4 This is a comparison chart of the change curves of coordinate error loss during the training process of this invention.

[0033] Figure 5 This is a comparison chart of the learning rate change curves of coordinate error during the training process of this invention.

[0034] Figure 6 This is a comparison chart of the angle error variation curves of coordinate errors during the training process of this invention.

[0035] Figure 7 This is a structural block diagram of the local terminal of an exemplary electronic device of the present invention.

[0036] Figure 8 This is a structural block diagram of the network terminal of an exemplary electronic device of the present invention. Detailed Implementation

[0037] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0038] In the description of this invention, it should be noted that the terms "preprocessing," "loading," "training," "testing," "channel," "space," "attention," "normalization," "loss," and "optimization," etc., indicate the operational or structural relationships based on the system flow or model structure relationships shown in the accompanying drawings. They are used only for the convenience of describing this invention and for simplifying the description, and do not indicate or imply that the operations or structures referred to must have a specific order or be constructed and operated in a specific manner. Therefore, they should not be construed as limitations on this invention. The terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, unless otherwise explicitly specified and limited, the terms "storage," "connection," "transmission," and "computation" should be interpreted broadly. For example, they can refer to direct storage or processing, or indirect storage or processing; they can refer to local computation or distributed computation; they can refer to fixed connections or configurable connections; they can refer to software implementation or hardware implementation. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0039] Example 1:

[0040] like Figures 1 to 6 As shown, an eye-tracking method based on a hybrid attention mechanism according to an embodiment of the present invention has the following process: Figure 1 As shown, it includes:

[0041] S1. Data Preprocessing: The original eye-tracking dataset is converted into a structured storage format. The mapping relationship between images and gaze coordinates is established through data parsing and alignment. Regular expressions are used to parse text files to extract frame identifiers and gaze coordinates. The left and right eye images and gaze coordinates that have passed the matching verification are stored in HDF5 format files according to the session-frame hierarchical structure. Compression algorithms are used to optimize storage space.

[0042] S2. Construct a deep neural network model with a hybrid attention mechanism. Use dual-channel input to process left and right eye images respectively. Design a basic feature extraction network containing multiple cascaded convolutional blocks. The number of feature map channels increases layer by layer while the spatial size decreases. Integrate channel attention module and spatial attention module to form a hybrid attention mechanism. The channel attention module focuses on important feature channels based on the Squeeze-and-Excitation mechanism. The spatial attention module generates spatial attention weights by calculating the statistical information of the feature map channel dimensions.

[0043] S3. Define a composite loss function based on geometric constraints, which includes pixel-level accuracy loss and three-dimensional geometric angle loss. The pixel-level accuracy loss uses the L1 loss function to calculate the absolute error between the predicted coordinates and the true coordinates. The three-dimensional geometric angle loss converts the two-dimensional screen gaze coordinates into a three-dimensional spatial vector and calculates the angle between the predicted vector and the true vector as the angle error.

[0044] S4. Efficient training and optimization strategies are adopted, including mixed precision training, dynamic learning rate scheduling, and early stopping strategy. Mixed precision training uses different precision calculations in forward and backward propagation to balance memory usage and numerical stability. Dynamic learning rate scheduling adjusts the learning rate during training to improve convergence speed. The early stopping strategy terminates training early based on the angle error of the validation set and saves the best model.

[0045] Figure 2 This is a schematic diagram of the hybrid attention neural network, showing the complete architecture of dual-channel input, basic convolutional block, attention module, and output layer. Here, Conv represents a convolutional layer, BaseConv represents a basic convolutional block, ReLU represents a non-linear activation function, CoordAtt represents a coordinate attention module, SE Layer represents a channel attention module, Dropout represents a random deactivation layer, Global Average Pooling represents global average pooling, and Linear represents a fully connected output layer used to map the extracted features to two-dimensional gaze coordinates.

[0046] Figure 4 This curve reflects the optimization of coordinate error during the training phase. The overall downward trend of the curve indicates that the model can gradually learn the mapping relationship between the left and right eye images and the gaze coordinates. The loss stabilizes and decreases in the later stages of training, indicating that the hybrid attention mechanism and composite loss function help improve the model's convergence performance.

[0047] Figure 5 This demonstrates the OneCycleLR dynamic learning rate scheduling strategy. The learning rate is gradually increased in the early stages of training to enhance parameter search and feature learning capabilities; then it gradually decreases, allowing the model to be refined and optimized with smaller steps in the later stages of training, thereby improving convergence stability.

[0048] Figure 6 This curve reflects the angular error between the predicted gaze direction and the actual gaze direction. The curve decreases and stabilizes in the later stages, indicating that the 3D geometric angle loss effectively constrains gaze direction deviation, making the model output more consistent with the angular accuracy requirements of eye-tracking tasks.

[0049] S1 specifically refers to:

[0050] 1A. Raw Data Parsing and Alignment: Extract left and right eye image files (.bmp format) and corresponding gaze coordinate text files (DownloadGaze*.txt format) from the PVeye eye-tracking dataset. Use regular expressions to parse the frame identifiers, horizontal gaze coordinates GX, and vertical gaze coordinates GY in the text files to establish a mapping dictionary from frame identifiers to two-dimensional gaze coordinates. At the same time, traverse the left and right eye image folders (named "session_numberL" and "session_numberR") and match and verify each image file with the corresponding gaze coordinates to ensure data consistency. Record and exclude missing or mismatched data.

[0051] 1B. Structured Storage and Compression Optimization: Validated image data and gaze coordinates are stored as HDF5 format files in a session-frame hierarchical structure. A group is created with the path "session_{session number} / frame_{frame identifier}". Within this group, two datasets, eye_L and eye_R, are created to store 224×224 pixel grayscale images of the left and right eyes, respectively. Two floating-point attributes, gaze_x and gaze_y, representing gaze coordinates are stored in the group attributes. The gzip compression algorithm is enabled, the data block size is set to (224, 224), and the image data is stored using the uint8 data type, achieving optimized storage space and efficient I / O access.

[0052] Specifically, the data preprocessing in S1 includes: The left and right eye images are used as two independent input channels to form an input tensor of shape [2, H, W]; 2 represents the number of input channels, the first channel is the left eye image, and the second channel is the right eye image; H represents the height of a single eye image in pixels; W represents the width of a single eye image in pixels. Normalize the input tensor.

[0053] S2 specifically refers to: 2A. Construction of Dual-Channel Input and Basic Feature Extraction Network: A convolutional neural network is constructed with dual-channel images as input. The first channel is the left-eye image, and the second channel is the right-eye image. The input tensor shape is [batch_size, 2, H, W], where batch_size represents the number of samples in each training batch, 2 represents the two input channels for the left and right eyes, H represents the image height, and W represents the image width. In this embodiment, the images are uniformly scaled to 224×224 pixels, so the input tensor can also be represented as [batch_size, 2, 224, 224]. A basic feature extraction module is designed, which contains 5 cascaded basic convolutional blocks (BaseConv). Each basic convolutional block consists of a 3×3 convolutional layer (stride of 2), a batch normalization layer, and a LeakyReLU activation function. The number of feature map channels increases by a factor of 1.5 after each convolutional block (from 16, 24, 36, 54 to 81), and the spatial size gradually decreases from 224×224 to 7×7.

[0054] 2B. Hybrid Attention Module Design and Integration: A channel attention module (SELayer) is designed based on the Squeeze-and-Excitation mechanism, containing a global average pooling layer, two fully connected layers, and a sigmoid activation function. The insertion position of this module in the network is dynamically controlled through configuration file parameters (Config.se_layers). A spatial attention module (SpatialAttention) is designed, which calculates the average and maximum values ​​of the feature maps in the channel dimension, concatenates them, and then generates spatial attention weights through a 7×7 convolutional layer and a sigmoid activation function. The channel attention module and the spatial attention module are integrated into the basic feature extraction network to form a hybrid attention mechanism. Finally, the features are mapped to two-dimensional gaze coordinates through a global average pooling layer and a fully connected layer.

[0055] The composite loss function in S3 includes: The L1 loss term calculates the absolute error between the predicted coordinates and the true coordinates. The angle loss term optimizes the conversion of two-dimensional screen coordinates into three-dimensional spatial angle errors.

[0056] S3 specifically refers to:

[0057] 3A. Pixel-level accuracy loss design: The L1 loss function is used to calculate the absolute error between the predicted gaze coordinates and the true gaze coordinates.

[0058] ;

[0059] in Let i be the gaze coordinates of the i-th sample predicted by the model. For the corresponding true gaze coordinates, For batch sizes, this loss provides pixel-level precision optimization;

[0060] 3B. Three-dimensional geometric angle loss design: Convert the two-dimensional screen gaze coordinates into three-dimensional spatial vectors, and calculate the angle between the predicted vector and the true vector as the angle error. :

[0061] ;

[0062] In the above formula, (x1, y1) are the true gaze coordinates, (x2, y2) are the predicted gaze coordinates, and z is the preset vertical distance parameter from the eye to the screen. This loss function makes the optimization target more consistent with the geometric characteristics of human vision.

[0063] S4 specifically refers to:

[0064] 4A. Mixed Precision Training and Gradient Optimization: Automatic Mixed Precision Training (AMP) is employed. During forward propagation, float16 precision is used to reduce memory usage, while during backpropagation, float32 precision is used to maintain numerical stability. A gradient scaler is used to prevent gradient underflow issues under float16 precision. The AdamW optimizer is used for parameter updates, with the weight decay parameter set to 1e. -5 To prevent overfitting;

[0065] 4B. Dynamic Learning Rate Scheduling and Early Stopping Strategy: A OneCycleLR learning rate scheduling strategy is adopted, linearly increasing the learning rate to its maximum value (1e) in the early stages of training. -4 During the training phase, a high learning rate is maintained, and the learning rate is decayed according to the cosine function in the later stage of training. An early stopping strategy is implemented based on the angle error of the validation set. The patience period is set to 10-20 training cycles. When the angle error of the validation set does not improve significantly within the patience period, the training is terminated early, and the model with the smallest angle error of the validation set is saved as the best model.

[0066] In the implementation, the hardware environment consisted of an NVIDIA GeForce RTX4070, an Intel Core i7 processor, and 16GB of RAM. The software environment included Python 3.8, PyTorch 1.12, and CUDA 11.6. The dataset used was the PVeye eye-tracking dataset, which contained left and right eye images and fixation coordinates from multiple subjects. The training parameters were set to a batch size of 512 and a learning rate of 1e. -4 Training rounds: 100; SE attention module insertion layer [1,3,5].

[0067] To verify the effectiveness of the method of this invention, the following comparison methods were set up: baseline CNN method (pure convolutional neural network without attention mechanism), SE-only attention method (using only channel attention module), spatial attention method (using only spatial attention module), and the method of this invention (hybrid attention mechanism, SE + spatial attention). Figure 4 As shown, the training loss fluctuated slightly in the early stages, then gradually decreased from a relatively high level to approximately 0.0277, indicating that the model can continuously reduce coordinate prediction error, and the hybrid attention structure is beneficial for extracting more effective binocular features. Figure 5 As shown, the learning rate first increases and then decreases according to a dynamic scheduling strategy. In the early stages, this helps the model escape unfavorable local regions, and in the later stages, it improves convergence stability by reducing the update step size. Figure 6 As shown, the angular error gradually decreased from approximately 1.0° to approximately 0.86°, indicating that the angular loss based on three-dimensional geometric constraints can directly improve the accuracy of line-of-sight direction estimation. (Summary) Figures 4 to 6 It can be seen that the method of the present invention can not only reduce the two-dimensional coordinate error, but also improve the angle error that is more in line with the needs of eye tracking applications, while the inference speed is maintained at about 120 frames / second, which meets the requirements of real-time eye tracking.

[0068] Example 2:

[0069] This invention also provides a system applicable to the eye-tracking method based on the hybrid attention mechanism described above, comprising: a data preprocessing module for converting raw eye-tracking data into a structured storage format; a data loading module for batch loading the preprocessed eye-tracking data; an eye-tracking model including a feature extraction network and a regression output layer, wherein the feature extraction network embeds a configurable channel attention module and a spatial attention module; a training optimization module for training the eye-tracking model; and a testing and evaluation module for evaluating the performance of the trained eye-tracking model.

[0070] Preferably, the data preprocessing module stores the raw data in HDF5 format, with a session-frame hierarchy, and each frame group contains a left-eye image, a right-eye image, and gaze coordinate attributes.

[0071] Preferably, the channel attention module includes: A global average pooling layer is used to generate channel description vectors; Two fully connected layers are used to learn the dependencies between channels; The sigmoid activation function is used to generate channel attention weights.

[0072] Preferably, the spatial attention module includes: a channel-dimensional average value calculation unit; a channel-dimensional maximum value calculation unit; a stitching layer for stitching the average value map and the maximum value map along the channel dimension; a convolutional layer for generating a spatial attention weight map; and a sigmoid activation function for generating spatial attention weights.

[0073] Example 3:

[0074] The present invention also discloses a machine-executable program that can be automatically executed by a machine to realize the eye-tracking method based on a hybrid attention mechanism as described above.

[0075] The machine (electronic device) mentioned above is, for example, a microcontroller, a single-board computer, a desktop computer, a laptop computer, a server, a programmable controller, or a field-programmable gate array.

[0076] Figure 7 This is a structural block diagram of the local terminal of an exemplary electronic device (machine) of the present invention; as shown... Figure 7 As shown, the electronic device of the present invention includes a processor 11, a memory 12, a storage space 13 for storing program code, and program code 14 for executing the method steps according to the present invention. The program code 14 for executing the method steps according to the present invention is used to execute the above-described control logic.

[0077] Figure 8 This is a structural block diagram of the network end of an exemplary electronic device of the present invention; as shown below. Figure 8 As shown, the present invention also provides an electronic device (machine), which may include at least one processor 210, at least one memory 230 communicatively connected to the processor, and a communication bus 240 and a communication interface 220 connecting different system components (including the memory 230 and the processor 210). The processor 210, the memory 230 and the communication interface 220 are connected through the communication bus 240 and communicate with each other. The communication interface 220 is used for data interaction with external devices. The memory 230 stores a machine-executable program that can be executed by the processor, and the processor 210 can execute the above-mentioned control logic by calling the machine-executable program.

[0078] Communication bus 240 represents one or more of several bus architectures, including a memory bus or memory controller, peripheral bus, graphics acceleration port, processor, or local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnection (PCI) bus.

[0079] Electronic devices typically include a variety of computer system readable media, which can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0080] Memory 230 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 230 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the control logic described above.

[0081] A program / utility having a set (at least one) of program modules can be stored in memory 230. Such program modules include, but are not limited to, an operating system, one or more applications, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0082] Machine-executable programs for performing this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, C++, and Python, and may also include specialized engineering languages ​​such as R. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0083] The present invention also discloses a storage medium on which a machine-executable program as described above is stored.

[0084] The aforementioned storage medium may be any combination of one or more computer-readable media. Computer-readable media may be, for example, computer-readable signal media or computer-readable storage media. Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium may be, for example, any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0085] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0086] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0087] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An eye-tracking method based on a hybrid attention mechanism, characterized in that, Includes the following steps: S1. Data preprocessing: The raw eye movement data is parsed and the coordinates are aligned. The frame identifier and gaze information are extracted by regular expression. The verified left and right eye images and gaze coordinates are stored in compressed HDF5 format according to the session-frame structure to achieve data structuring and efficient storage. S2. Construct a deep neural network model with a hybrid attention mechanism. Use dual-channel input to process left and right eye images respectively. Design a basic feature extraction network containing multiple cascaded convolutional blocks. The number of feature map channels increases layer by layer while the spatial size decreases. Integrate channel attention module and spatial attention module to form a hybrid attention mechanism. The channel attention module focuses on important feature channels based on the Squeeze-and-Excitation mechanism. The spatial attention module generates spatial attention weights by calculating the statistical information of the feature map channel dimensions. S3. Define a composite loss function based on geometric constraints, which includes pixel-level accuracy loss and three-dimensional geometric angle loss. The pixel-level accuracy loss uses the L1 loss function to calculate the absolute error between the predicted coordinates and the true coordinates. The three-dimensional geometric angle loss converts the two-dimensional screen gaze coordinates into a three-dimensional spatial vector and calculates the angle between the predicted vector and the true vector as the angle error. S4. Efficient training and optimization strategies are adopted, including mixed precision training, dynamic learning rate scheduling, and early stopping strategy. Mixed precision training uses different precision calculations in forward and backward propagation to balance memory usage and numerical stability. Dynamic learning rate scheduling adjusts the learning rate during training to improve convergence speed. The early stopping strategy terminates training early based on the angle error of the validation set and saves the best model.

2. The method according to claim 1, characterized in that, The data preprocessing in S1 specifically includes: The left and right eye images are used as two independent input channels to form an input tensor of shape [2, H, W]; 2 represents the number of input channels, the first channel is the left eye image, and the second channel is the right eye image; H represents the height of a single eye image in pixels; W represents the width of a single eye image in pixels. Normalize the input tensor.

3. The method according to claim 1, characterized in that, The insertion position of the channel attention module in S2 is dynamically specified through a configuration file, supporting selective insertion after different convolutional layers of the feature extraction network.

4. The method according to claim 1, characterized in that, The composite loss function in S3 includes: The L1 loss term calculates the absolute error between the predicted coordinates and the true coordinates. The angle loss term optimizes the conversion of two-dimensional screen coordinates into three-dimensional spatial angle errors.

5. The method according to claim 4, characterized in that, The angle error in S3: ； In the above formula, (x1,y1) are the actual gaze coordinates, (x2,y2) are the predicted gaze coordinates, and z is the preset vertical distance parameter from the eyeball to the screen.

6. The method according to claim 1, characterized in that, The efficient training and optimization strategy in S4 shall employ at least one of the following: Mixed-precision training, using float16 precision for forward propagation and gradient calculation; and / or Dynamic learning rate scheduling, employing the OneCycleLR strategy; and / or Use the AdamW optimizer to update parameters.

7. A system applicable to the eye-tracking method based on a hybrid attention mechanism as described in any one of claims 1-6, characterized in that, include: The data preprocessing module is used to convert raw eye-tracking data into a structured storage format; The data loading module is used to load preprocessed eye-tracking data in batches. An eye-tracking model includes a feature extraction network and a regression output layer, wherein the feature extraction network embeds a configurable channel attention module and a spatial attention module; The training optimization module is used to train the eye-tracking model; The testing and evaluation module is used to evaluate the performance of the trained eye-tracking model.

8. The system according to claim 7, characterized in that, The data preprocessing module stores the raw data in HDF5 format, with a session-frame hierarchy. Each frame group contains the left-eye image, the right-eye image, and gaze coordinates.

9. The system according to claim 7, characterized in that, The channel attention module includes: A global average pooling layer is used to generate channel description vectors; Two fully connected layers are used to learn the dependencies between channels; The sigmoid activation function is used to generate channel attention weights.

10. The system according to claim 7, characterized in that, The spatial attention module includes: a channel-dimensional average value calculation unit; a channel-dimensional maximum value calculation unit; a stitching layer for stitching the average value map and the maximum value map along the channel dimension; a convolutional layer for generating a spatial attention weight map; and a sigmoid activation function for generating spatial attention weights.

Images