Lightweight gaze estimation method based on coordinate attention and face redundancy compression

Abstract

The present application relates to the technical field of computer vision, and particularly relates to a lightweight gaze estimation method based on coordinate attention and face redundancy compression, comprising the following steps: obtaining a face image, a left eye image, a right eye image and head pose information; inputting the left eye image and the right eye image into an eye branch sharing weights to output fused features of both eyes; the eye branch is constructed based on a MobileNetV4 backbone network, and a coordinate attention module is arranged at a shallow feature stage of the eye branch; inputting the face image into a face branch to output face features; the face branch is constructed based on a MobileNetV4 backbone network, and a face redundancy compression module is arranged at an early down-sampling stage of the face branch; fusing the fused features of both eyes, the face features and the head pose information to obtain fused features; inputting the fused features into a regression head to predict a two-dimensional gaze angle, and high-precision two-dimensional gaze angle estimation is realized under the premise of maintaining a low parameter amount and a floating point operation number.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a lightweight gaze estimation method based on coordinate attention and facial redundancy compression. Background Technology

[0002] Gaze estimation aims to infer a person's gaze direction based on facial or eye features. It is an important visual perception technology for understanding human visual attention and behavioral intentions, and is widely used in human-computer interaction, driver monitoring, smart cockpits, and mobile visual perception. With the widespread use of RGB cameras and mobile visual sensing devices, appearance-based gaze estimation methods have become the mainstream technology in practical applications due to their advantages of not requiring infrared light sources and complex geometric modeling, and being able to directly learn gaze direction mapping relationships from natural images.

[0003] In recent years, models based on convolutional neural networks and Transformers have significantly improved the accuracy of gaze estimation. Numerous methods have enhanced their ability to model subtle changes in eye appearance, head pose differences, and individual facial features by employing deeper neural network architectures, more complex feature fusion modules, or larger-scale training data. However, these high-precision models typically have millions or even hundreds of millions of parameters, high computational costs, and large memory footprints, making them difficult to deploy in real-time on resource-constrained mobile devices, embedded systems, and automotive platforms. In practical scenarios such as driver monitoring and mobile human-computer interaction, gaze estimation models not only need high prediction accuracy but also must possess low latency, small size, and low power consumption.

[0004] To reduce model complexity, some studies have attempted to replace traditional VGG and ResNet backbones with lightweight networks such as MobileNet and GhostNet. However, gaze estimation differs from general image classification: determining the gaze direction depends on both subtle appearance changes in the eye region (such as eyelid, corner of the eye, and pupil texture) and overall pose cues in the facial region. Lightweight networks have a limited number of channels, making them prone to losing important spatial location information during downsampling, leading to a significant decrease in accuracy. Furthermore, facial images contain a large number of redundant features weakly correlated with gaze estimation (such as background and skin texture); indiscriminate extraction of these features increases computational burden and introduces noise.

[0005] Therefore, there is an urgent need for a lightweight gaze estimation method that can maintain high gaze estimation accuracy while having low parameter count and low computational complexity, in order to adapt to resource-constrained visual perception scenarios. Summary of the Invention

[0006] The purpose of this invention is to address the technical problems of high computational resource consumption and severe accuracy loss of lightweight networks in existing gaze estimation methods, and to provide a lightweight gaze estimation method based on coordinate attention and facial redundancy compression, which achieves high-precision two-dimensional gaze angle estimation while maintaining a low number of parameters and floating-point operations.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A lightweight gaze estimation method based on coordinate attention and facial redundancy compression includes the following steps:

[0009] Acquire face image, left eye image, right eye image, and head pose information;

[0010] The left and right eye images are input into an eye branch with shared weights, and the binocular fusion features are output; wherein, the eye branch is constructed based on the MobileNetV4 backbone network and a coordinate attention module is set in its shallow feature stage;

[0011] The face image is input into a face branch, and the face features are output; the face branch is constructed based on the MobileNetV4 backbone network and has a face redundancy compression module set in its early downsampling stage.

[0012] The fusion feature is obtained by fusing the binocular fusion features, facial features, and head posture information;

[0013] The fused features are input into a regression head to predict the two-dimensional gaze angle.

[0014] A further improvement is that the facial redundancy compression module is composed of downsampling convolution, spatial reconstruction unit and channel reconstruction unit connected in series, which is used to suppress facial and background responses that are not related to gaze.

[0015] A further improvement is that the downsampling convolution uses a 3×3 convolution with a stride of 2 to spatially downsample and expand the channels of the input shallow facial features; the spatial reconstruction unit is used to divide the input feature map into several groups along the channel dimension, calculate the similarity between different spatial locations within each group, and filter and merge redundant spatial locations through a threshold to reduce spatial redundancy responses; the channel reconstruction unit is used to divide the input feature map into several groups along the channel dimension, perform intra-group information fusion through 1×1 convolution, and then weight each channel with channel attention weights generated by global average pooling to suppress redundant channel responses; the output feature map of the facial redundancy compression module is used as the input of subsequent MobileNetV4 layers.

[0016] A further improvement is that the coordinate attention module is used to encode orientation-aware spatial location information along the height and width directions respectively, and to reweight shallow features by generating height direction attention weights and width direction attention weights.

[0017] A further improvement lies in the fact that the specific processing procedure of the coordinate attention module includes:

[0018] Let the shallow features input to the coordinate attention module be... ,in For the number of channels, For feature map height, The width of the feature map;

[0019] Perform average pooling along the height direction for each channel to obtain the height direction descriptor. , its first Each component is defined as:

[0020]

[0021] in , , ;

[0022] Perform average pooling along the width direction for each channel to obtain the width direction descriptor. , its first Each component is defined as:

[0023]

[0024] in , ;

[0025] The height and width descriptors are concatenated along the spatial dimensions, and then sequentially processed through a 1×1 convolution, batch normalization, and ReLU activation function to obtain intermediate features. ; the intermediate features Segmented into two independent directional features along the channel dimension and ;Will and The height-oriented attention weights are generated by passing the weights through a 1×1 convolution and a sigmoid function. Attention weights in the width direction ;

[0026] Output feature map The Each channel is located in The value at that location is:

[0027]

[0028] in The first is the height-oriented attention weight. Channel 1 The value of the row, Attention weights in the width direction Channel 1 The value of the column.

[0029] A further improvement is that the binocular fusion feature is obtained by concatenating the left-eye and right-eye features output from the eye branches into vectors to obtain the concatenated feature. Then, the concatenated feature is input into a fully connected layer and projected to obtain the binocular fusion feature.

[0030] The facial features are output by the last fully connected layer of the facial branch;

[0031] The head attitude information is a two-dimensional vector containing the head pitch angle and yaw angle;

[0032] The fusion feature is a vector obtained by concatenating the binocular fusion feature, facial features, and head pose information.

[0033] A further improvement is that the regression head is composed of a first fully connected layer and a second fully connected layer connected in series. The input of the first fully connected layer is the fused feature, and the output is activated by the ReLU activation function. The second fully connected layer outputs a two-dimensional gaze angle, which includes a predicted pitch angle and a predicted yaw angle.

[0034] A further improvement is that the method also includes a training step:

[0035] Obtain a training sample set, each training sample containing a face image, a left eye image, a right eye image, a two-dimensional head pose vector, and a real two-dimensional gaze angle as a supervision signal;

[0036] After the training samples undergo feature extraction, fusion, and prediction steps, the predicted two-dimensional gaze angle is obtained.

[0037] The loss between the predicted 2D gaze angle and the actual 2D gaze angle is calculated using the mean squared error loss function. The network parameters of the eye branch, face branch, and regression head are updated using the backpropagation algorithm. The above process is repeated until the network converges.

[0038] A further improvement involves data augmentation of the left-eye and right-eye images in the training samples during the training process: a left-eye image or a right-eye image is randomly selected with a preset probability, and one of the following perturbations is randomly applied to the selected eye image:

[0039] Random occlusion: Placing a black square at a random location in the image;

[0040] Gaussian blur: blurs an image using a Gaussian kernel;

[0041] Brightness adjustment: Multiplies the image brightness by a coefficient;

[0042] Slight geometric transformation: Randomly rotate by a small angle or translate horizontally / vertically by a few pixels.

[0043] A further improvement is that the MobileNetV4 backbone network adopts the MobileNetV4ConvSmall structure, which includes depthwise separable convolution and a general inverted bottleneck module. The depthwise separable convolution decomposes the standard convolution into channel-wise spatial convolution and pointwise convolution to reduce the number of parameters and computation. The general inverted bottleneck module realizes spatial feature extraction and channel interaction through configurable depthwise convolution.

[0044] The beneficial effects of this invention are as follows:

[0045] This invention introduces a coordinate attention module into the shared eye branch, which can encode spatial position information of directional perception along the height and width directions, effectively capturing subtle appearance changes in areas such as the corner of the eye, eyelids, and pupils, overcoming the problem of spatial information loss in lightweight networks, and significantly improving gaze estimation accuracy.

[0046] This invention introduces a facial redundancy compression module into the facial branch, which suppresses facial and background responses that are not related to gaze through spatial reconstruction and channel reconstruction, so that the facial branch can focus more on head posture and gaze-related structural information, thereby improving the efficiency and effectiveness of facial feature extraction.

[0047] This invention achieves high accuracy while maintaining low computational overhead by fusing multi-source features (eyes, face, head pose) and predicting gaze angles with a lightweight regression head. It has achieved competitive results on the MPIIFaceGaze, EyeDiap, and GazeCapture datasets, achieving an excellent balance between accuracy and efficiency. Attached Figure Description

[0048] Figure 1 This is a diagram showing the overall architecture of the lightweight multi-branch gaze estimation network of this invention.

[0049] Figure 2 This is a flowchart of a lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to the present invention.

[0050] Figure 3 This is a schematic diagram of the coordinate attention module in the shared eye branch of the present invention;

[0051] Figure 4 This is a schematic diagram of the facial redundancy compression module in the facial branch of the present invention;

[0052] Figure 5 A comparison of the visualization of facial branch activation responses. Detailed Implementation

[0053] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0054] Please refer to the attached document. Figure 1 To be continued Figure 5 This invention proposes a lightweight gaze estimation method based on coordinate attention and facial redundancy compression. The application scenario of this embodiment is a real-time gaze interaction system for a smartphone's front-facing camera. This system includes: an RGB camera (640×480 resolution, 30fps), a lightweight face detection module (for locating the face and left and right eye regions), a head pose estimation module (based on face keypoint PnP calculation), and a mobile AI processor (e.g., the NPU built into a Qualcomm Snapdragon 8 series chip) integrating the lightweight multi-branch gaze estimation network of this invention (named LightGazeNet). In this embodiment, all images are uniformly scaled to 224×224 pixels and standardized before being input to the network.

[0055] like Figure 1 As shown, the lightweight multi-branch gaze estimation network constructed in this invention includes: a shared-weight eye branch network for extracting local eye details, a facial branch network for capturing global facial structure information, a multi-source feature fusion module for multi-source feature fusion, and a lightweight regression head. The network uses face images... Left eye image Right eye image and two-dimensional head pose vector As input, output two-dimensional gaze angle ,in The predicted pitch angle. This is the predicted yaw angle.

[0056] like Figure 2 As shown, the lightweight gaze estimation method based on coordinate attention and facial redundancy compression proposed in this embodiment of the invention specifically includes the following steps:

[0057] Step S1: Obtain face image, left eye image, right eye image and head pose information.

[0058] Step S2: Input the left eye image and the right eye image into an eye branch with shared weights, and output the binocular fusion features; the eye branch is constructed based on the MobileNetV4 backbone network, and a coordinate attention module is set in its shallow feature stage.

[0059] Step S3: Input the face image into a face branch and output the face features; the face branch is constructed based on the MobileNetV4 backbone network and has a face redundancy compression module set in its early downsampling stage.

[0060] Step S4: Fuse the binocular fusion features, facial features, and head posture information to obtain the fusion features.

[0061] Step S5: Input the fused features into a regression head to predict the two-dimensional gaze angle.

[0062] The embodiments of the present invention will now be described in more detail:

[0063] Specifically, in this embodiment, MobileNetV4ConvSmall is used as the basic feature extraction backbone network, and its core components include depthwise separable convolution and a general inverted bottleneck structure. Depthwise separable convolution decomposes standard convolution into channel-wise spatial convolution and pointwise convolution, significantly reducing the number of parameters and computational cost. Let the input feature map spatial size be... ( (Where the width and height of the feature map are in pixels), and the number of input channels is... The standard convolutional kernel size is DK× ( (where the width and height are the convolution kernel widths and heights), and the number of output channels is... The number of parameters in a standard convolution is... and computational complexity (FLOPs) are as follows:

[0064]

[0065]

[0066] After using depthwise separable convolution, the number of parameters is reduced. and computational complexity It becomes:

[0067]

[0068]

[0069] In the above formula, This is the depthwise convolution part (convolution is performed separately for each input channel). This is the pointwise convolution part (1×1 convolutional mixing channel). It's evident that depthwise separable convolution decouples spatial filtering from channel mixing, avoiding the high computational overhead caused by coupling each output channel to a complete spatial convolution kernel. Significant savings are achieved when the size of the parameter is large. This design allows the backbone network of this invention to significantly reduce the number of parameters and computational complexity while maintaining high feature extraction capabilities.

[0070] Specifically, the eye branch uses a truncated MobileNetV4ConvSmall network, with the left and right eyes sharing the same network parameters. The specific structure is shown in Table 1.

[0071] enter Left / Right Eye Images 3 × 224 × 224 Conv-BN-ReLU6 3×3 standard convolution, stride 2, output channels 32 32 × 112 × 112 MobileNet V4 Layer 1 Depthwise separable convolutional blocks with 32 output channels 32 × 56 × 56 CoordAtt (Coord Attention Module) Direction awareness and attention, maintaining resolution 32 × 56 × 56 MobileNet V4 Layer 2 Depthwise separable convolutional blocks with 64 output channels 64×28×28 MobileNet V4 Layer 3 Depthwise separable convolutional blocks with 96 output channels 96 × 14 × 14 Global average pooling layer Spatial dimension compressed to 1×1 96×1×1 Fully connected layer Linear projection, output 128 dimensions 128

[0072] Table 1 LightGazeNet Eye Branch Configuration

[0073] Define the eye branch as a function Then the left eye features Right eye features The two feature vectors are concatenated and then input into the eye fusion layer. (A fully connected layer with an input dimension of 256 and an output dimension of 128) yields binocular fusion features:

[0074]

[0075] in, Indicates the fusion layer of the eye. This indicates a vector concatenation operation.

[0076] Specifically, such as Figure 3As shown, the coordinate attention module is used to enhance spatial feature modeling for orientation awareness. In lightweight gaze estimation networks, directly using a compressed lightweight backbone network leads to a decrease in gaze estimation accuracy. The fundamental reason is that lightweight networks have limited channel capacity, and repeated downsampling results in the loss of subtle spatial location information in the eye region. Minor deformations in areas such as the eyelids, corners of the eyes, and pupils are closely related to changes in gaze direction, and this information is easily weakened as spatial resolution gradually decreases. To address this issue, this invention introduces a coordinate attention module in the shallow feature stage of the eye branch. The core idea of ​​the coordinate attention module is to aggregate feature maps along both height and width directions, capturing long-distance dependencies in orientation awareness while preserving accurate spatial location encoding. Unlike traditional channel attention, coordinate attention not only generates channel weights but also row and column attention weights related to spatial coordinates, enabling the network to pinpoint which features are more important.

[0077] Specifically, in this invention, the coordinate attention module is used to encode orientation-aware spatial location information along the height and width directions respectively, and to reweight shallow features by generating height-direction attention weights and width-direction attention weights. The specific processing procedure of the coordinate attention module is as follows:

[0078] Let the input shallow features be ,in For the number of channels, The feature map height (in pixels). This is the feature map width (in pixels). In this embodiment... , First, average pooling is performed on each channel along both the height and width directions to obtain two directional descriptors:

[0079] Height direction descriptor , its first Each component is defined as:

[0080]

[0081] in, Indicates the first Position in each channel pixel values, , , .

[0082] Width direction descriptor , its first Each component is defined as:

[0083]

[0084] in , .

[0085] By concatenating the two directional descriptors along the spatial dimensions, we obtain... Then through a shared 1×1 convolution. (Compress the number of channels to) In this embodiment, the compression ratio The intermediate features are then obtained through batch normalization and ReLU activation. :

[0086]

[0087] in It is the ReLU activation function. This is for batch normalization operations.

[0088] Will Segmented into two independent directional features along the channel dimension and Then, each is processed through two independent 1×1 convolutions. and Restore the number of channels to Then, the height-oriented attention weights are generated using the Sigmoid function. Attention weights in the width direction :

[0089]

[0090]

[0091] in This represents the Sigmoid function, with an output range of (0,1).

[0092] Final output feature map The Each channel is located in The value is obtained by multiplying the input features by the attention weights in both directions:

[0093]

[0094] in The first is the height-oriented attention weight. Channel 1 The value of the row, Attention weights in the width direction Channel 1 The value of the column.

[0095] Understandably, through the process described above, the coordinate attention module enables the network to capture long-range dependencies in both horizontal and vertical directions while preserving precise positional information, thereby enhancing its sensitivity to subtle changes in areas such as the corners of the eyes, eyelids, and pupils. The coordinate attention module introduces only a small amount of additional computation to significantly improve the gaze estimation accuracy of lightweight networks.

[0096] Specifically, the face branch is also based on MobileNetV4ConvSmall, but a Face Redundancy Compression Module (FRCM) is introduced in the early downsampling stage. The specific structure is shown in Table 2.

[0097] enter Face image 3 × 224 × 224 Conv-BN-ReLU6 3×3 standard convolution, stride 2, output channels 32 32 × 112 × 112 MobileNet V4 Layer 1 Depthwise separable convolutional blocks with 32 output channels 32 × 56 × 56 Facial Redundancy Compression Module (FRCM) Downsampling convolution + spatial reconstruction + channel reconstruction 64×28×28 MobileNet V4 Layer 3 Depthwise separable convolutional blocks with 96 output channels 96×14×14 MobileNet V4 Layer 4 Depthwise separable convolutional blocks, 128 output channels 128 × 14 × 14 Global average pooling layer Spatial dimension compressed to 1×1 128×1×1 Fully connected layer Linear projection, output 64 dimensions 64

[0098] Table 2 LightGazeNet Facial Branch Configuration

[0099] like Figure 4 As shown, the Face Redundancy Compression Module (FRCM) consists of downsampling convolutions. It consists of a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU) connected in series, used to suppress facial and background responses that are not related to gaze.

[0100] Let the input shallow surface features be First, spatial downsampling and channel expansion are performed using a 3×3 convolution with a stride of 2:

[0101]

[0102] in It uses 3×3 convolution with a stride of 2 and 64 output channels, followed by batch normalization and ReLU.

[0103] Then Passing through the spatial reconstruction unit and the channel reconstruction unit in sequence:

[0104]

[0105]

[0106] The operation procedure for the Spatial Reconfiguration Unit (SRU) is as follows: Divided along the channel dimension into Group (in this embodiment) Within each group, the similarity between spatial locations is calculated. Redundant spatial locations are filtered and merged using a threshold. The spatial size of the output feature map remains unchanged, but the number of channels may vary. The operation of the Channel Reconstruction Unit (CRU) is as follows: the input feature map is divided into several groups along the channel dimension. Within each group, information is fused using 1×1 convolution. Then, channel attention weights generated by global average pooling are used to weight the channels to suppress redundant channels. The final output... The size is 64×28×28, which is used as the input for the subsequent MobileNetV4 layer.

[0107] The principle behind the facial redundancy compression module is understandable: Besides containing gaze-related structural information (such as head orientation, facial contours, and the relative position of the eyes within the face), facial images also contain a large amount of redundant responses weakly related to gaze estimation, such as background regions, skin texture, non-discriminatory facial regions (such as cheeks and forehead), and interference caused by lighting variations. If this redundant information is directly extracted by a lightweight convolutional network, it will not only increase the computational burden but also introduce noise during the feature fusion stage, interfering with the final gaze regression. This invention introduces a facial redundancy compression module in the early downsampling stage of the facial branch. While the feature map still has high spatial resolution, it actively suppresses redundant responses through two mechanisms: spatial reconstruction and channel reconstruction. The spatial reconstruction unit reduces redundant spatial responses and encourages the facial branch to focus on facial structural regions related to head posture and gaze direction. The channel reconstruction unit further compresses and reorganizes channel responses, enhancing the interaction between information channels while suppressing redundant channel activation. This allows the network to focus more on gaze-related regions, preventing responses with low information content from propagating to deeper layers, thereby improving facial feature extraction efficiency.

[0108] The facial branch ultimately outputs 64-dimensional facial features:

[0109] Specifically, in this embodiment, the head pose information can be obtained in the following way: facial key point detection is performed on the face image to obtain the two-dimensional coordinates of at least a number of key points including the tip of the nose, the corner of the eye, and the corner of the mouth; the PnP algorithm is used to solve the rotation matrix and translation vector between the two-dimensional key points and the corresponding points of the three-dimensional face model, and the head pitch angle and yaw angle are calculated from the rotation matrix to form a two-dimensional head pose vector.

[0110] Specifically, after obtaining the binocular fusion features, facial features, and two-dimensional head pose vectors, the binocular fusion features are... (128 dimensions), facial features (64-dimensional) and two-dimensional head pose vector (2D) concatenated into a 194-dimensional fused feature vector:

[0111]

[0112] The head pose vector , Head pitch angle (unit: radians). The yaw angle is the nose angle (in radians).

[0113] Understandably, head pose information, as a geometric prior, provides complementary clues for gaze estimation. When the head rotates, the relative position of the eyeball in the eye socket changes. It is difficult to accurately estimate the absolute gaze direction by relying solely on the appearance of the eye. The introduction of head pose helps to decouple head movement and eye movement.

[0114] Specifically, in this embodiment, the lightweight regression head It consists of two fully connected layers:

[0115] The first fully connected layer has an input dimension of 194 and an output dimension of 64, followed by a ReLU activation function and Dropout (ratio 0.2).

[0116] The second fully connected layer has an input dimension of 64 and an output dimension of 2 (pitch and yaw).

[0117] The final predicted two-dimensional gaze angle is:

[0118]

[0119] In this embodiment, the method further includes a training step:

[0120] Obtain a training sample set, each training sample containing a face image, a left eye image, a right eye image, a two-dimensional head pose vector, and a real two-dimensional gaze angle as a supervision signal.

[0121] After the training samples undergo feature extraction, fusion, and prediction steps, the predicted two-dimensional gaze angle is obtained.

[0122] The loss between the predicted 2D gaze angle and the actual 2D gaze angle is calculated using the mean squared error loss function. The network parameters of the eye branch, face branch, and regression head are updated using the backpropagation algorithm. The above process is repeated until the network converges.

[0123] Specifically, this embodiment uses the following three public datasets for training and evaluation:

[0124] MPIIFaceGaze contains 213,659 facial images from 15 subjects, providing paired left-eye, right-eye, full-face cropped images, head pose vectors, and 3D gaze vectors. Leave-one-out cross-validation is used: one subject is selected as the test set each time, and the remaining 14 subjects are used as the training set (of which 10% are randomly selected as the validation set).

[0125] EyeDiap: Includes RGB-D recordings of 16 subjects across three visual tasks, covering both static and dynamic head movements. Four-fold cross-validation was used to evaluate the model's generalization ability under different head movement conditions.

[0126] GazeCapture: Contains approximately 2.5 million images from 1471 subjects, captured by the front-facing cameras of mobile phones and tablets. A subject-independent partitioning method was used: 1271 subjects for training, 50 for validation, and 150 for testing. Gaze labels are represented as two-dimensional gaze positions (in centimeters) relative to the camera center.

[0127] Understandably, these datasets are complementary in terms of data size, acquisition environment, and annotation format. MPIIFaceGaze is primarily used for subject-independent 3D gaze direction estimation; GazeCapture enables large-scale 2D gaze position estimation on mobile devices; and EyeDiap is used to evaluate the model's generalization ability under static and dynamic head motion conditions.

[0128] First, preprocessing is required. The preprocessing steps are as follows: Based on the facial key points provided in the dataset, crop out the face region (boundary box enlarged by 1.2 times), the left eye region, and the right eye region (boundary box enlarged by 1.5 times); scale all images to 224×224 pixels using bilinear interpolation; standardize the images. The mean =[0.485, 0.456, 0.406] (corresponding to the three RGB channels), standard deviation =[0.229,0.224,0.225]; The head pose vector directly uses the values ​​provided by the dataset (normalized to the range of [−1,1].

[0129] Online data augmentation was used during training. The augmentation strategy was as follows: the left-eye image or the right-eye image was randomly selected with a probability of 0.5, and then one of the following four perturbations was applied to the image of that eye (selected with equal probability):

[0130] Random occlusion: Place a 20×20 pixel black square at a random location in the image.

[0131] Gaussian blur: using a kernel size of 5 and a standard deviation of 1 / 2. Blur using a Gaussian kernel of 1.0.

[0132] Brightness adjustment: Multiply the image brightness by a coefficient that is uniformly and randomly sampled in the range [0.7, 1.3].

[0133] Slight geometric transformations: random rotation ±5∘, horizontal / vertical translation ±5 pixels.

[0134] Understandably, the above enhancement strategies simulate monocular occlusion, motion blur, lighting changes, and pose changes in real-world scenarios, enhancing the model's robustness to non-ideal acquisition conditions while keeping the gaze label unchanged.

[0135] In this embodiment, mean squared error (MSE) loss is used as the supervision signal for gaze direction regression:

[0136]

[0137] in Batch size (in this example) =16), For the predicted two-dimensional gaze angle vector , The corresponding true gaze angle vector , Represents the L2 norm, i.e. .

[0138] The optimizer uses AdamW with an initial learning rate of Weight decay A cosine annealing learning rate scheduling strategy is employed, comprising three warmup epochs. Batch size... =16. The model was trained for 25 epochs on the MPIIFaceGaze dataset and 18 epochs on other datasets. After each epoch, the optimal checkpoint was selected based on the average angle error on the validation set. The model convergence criterion was: the validation set loss decreased by less than 16% for 10 consecutive epochs. The degree, or reaching the maximum number of training rounds.

[0139] The evaluation metrics are as follows: For MPIIFaceGaze and EyeDiap, the average angle error (degrees) is used as the evaluation metric. For each test sample, the predicted pitch angle is... and yaw angle Convert to a 3D unit gaze vector:

[0140]

[0141] True gaze vector Similarly, the angle error for each sample is:

[0142]

[0143] in The dot product operation of vectors results in the following average angular error:

[0144]

[0145] For GazeCapture, the Euclidean distance (in centimeters) between the predicted gaze position and the actual gaze position is used as the evaluation metric:

[0146]

[0147] in The predicted two-dimensional gaze position (in centimeters). This represents the true 2D gaze position. Errors for the mobile phone and tablet subsets are reported separately.

[0148] To verify the effectiveness of this invention, experiments were conducted on the three datasets mentioned above, and comparisons were made with various advanced methods. The experimental hardware platform consisted of an Intel Core i9-10900K CPU, an NVIDIA RTX 3090 GPU, and PyTorch 1.10. Model inference time was further tested on a mobile phone equipped with a Qualcomm Snapdragon 865 mobile platform (single frame, batch size 1, deployed using TFLite INT8 quantization).

[0149] Table 3 presents a quantitative comparison of LightGazeNet and representative methods on three datasets. Among them, iTracker, CA-Net, and L2CS-Net are CNN-based methods, while SUGE uses a hybrid CNN+Transformer architecture.

[0150]

[0151] Table 3

[0152] Table 3 shows the comparison with state-of-the-art methods on a common gaze estimation dataset (where MPIIFaceGaze and EyeDiap are in degrees, and GazeCapture is in centimeters). The results show that the average angle error of the present invention on MPIIFaceGaze is 3.77°, which is better than all the comparison methods except AFF-Net (3.90°), and the number of parameters is much smaller than AFF-Net (1.54M for the present invention, compared to approximately 5M for AFF-Net). The present invention achieves the best result of 1.55cm on the mobile phone subset and 2.42cm on the tablet subset of GazeCapture, outperforming all the comparison methods. The present invention achieves 5.12° on EyeDiap, which is comparable to CA-Net (5.27°), L2CS-Net (5.24°), and SUGE (5.04°), with a significantly reduced computational cost. These results demonstrate that the present invention achieves competitive accuracy while maintaining extremely low computational cost, i.e., it achieves a good balance between estimation accuracy and computational efficiency, which is particularly important for resource-constrained visual perception applications.

[0153] To quantitatively analyze the contributions of the CoordAtt module and the Face Redundancy Compression module (FRCM), this invention conducts ablation experiments on the MPIIFaceGaze and EyeDiap datasets. The baseline model has the same multi-branch structure but does not include CoordAtt and FRCM, using only the original MobileNetV4 backbone. Experimental results are shown in Table 4.

[0154] MobileNetV4 4.12° 5.56° MobileNetV4+CoordAtt 3.89° 5.38° MobileNetV4+FRCM 3.85° 5.33° MobileNetV4+CoordAtt+FRCM 3.77° 5.12°

[0155] Table 4

[0156] Ablation experiments show that adding CoordAtt alone reduces the average angular error of MPIIFaceGaze by 0.23° (from 4.12° to 3.89°) and EyeDiap by 0.18° (from 5.56° to 5.38°), validating the effectiveness of orientation-aware attention in modeling subtle eye features. Adding FRCM alone reduces the error of MPIIFaceGaze by 0.27° (to 3.85°) and EyeDiap by 0.23° (to 5.33°), demonstrating that suppressing redundant facial responses helps extract global structural information. The combined use of both modules achieves the best performance, reducing the error by 0.35° and 0.44° from the baseline on the two datasets, respectively, validating the complementarity between the modules. Ablation experiments demonstrate that both task-specific modules significantly contribute to the final performance, and their combined use yields the best results.

[0157] Table 5 compares the number of parameters, FLOPs, and inference time on mobile devices of the present invention with those of representative methods.

[0158] iTracker CNN 2.52 0.88 – AGE-Net Transformer 109.0 35.70 66.00 RT-GENE CNN 30.0 30.80 35.00 CA-Net CNN 34.0 15.60 30.00 L2CS-Net CNN 23.52 16.53 – Dilated-Net Dilated-CNN 3.90 3.10 6.70 Ours (This invention) CNN 1.54 0.54 8.47

[0159] Table 5

[0160] The results show that this invention requires only 1.54M parameters and 0.54G FLOPs, significantly lower than all comparable methods. Compared to iTracker, the number of parameters and FLOPs are reduced by approximately 39%, while accuracy is improved. Compared to the equally lightweight Dilated-Net, the number of parameters is reduced by approximately 60%, FLOPs by approximately 83%, and accuracy is improved (Dilated-Net has an error of approximately 4.5° on MPIIFaceGaze, which is better than the 3.77° of this invention). The inference time of this invention is 8.47ms, slightly higher than Dilated-Net's 6.7ms, but the invention has higher accuracy and fewer parameters. Overall, this invention achieves an excellent balance between accuracy and efficiency.

[0161] To intuitively understand the role of the facial redundancy compression module, this invention extracts the feature maps of the facial branches before the third layer of MobileNetV4 (i.e., the features after the FRCM output). The heatmap is generated by averaging the channel dimensions and then overlaid on the original face image. The comparison model is a version with FRCM removed (i.e., the baseline model directly uses downsampling convolutions without SRU and CRU). Samples with different head poses and lighting conditions from MPIIFaceGaze are selected for visualization. Visualization of facial branch activation responses is shown below. Figure 5 As shown, for each sample, each column displays the input image, the activation map without FRCM, and the activation map after using FRCM, with warmer colors indicating stronger responses.

[0162] Visualization results show that without FRCM, the heatmap response is scattered across the entire facial region, including irrelevant areas such as the cheeks, forehead, and background. However, with FRCM, the heatmap response is significantly concentrated around the eyes, the brow, and the bridge of the nose; these areas are highly correlated with gaze direction and head posture. This qualitatively validates that FRCM effectively suppresses redundant facial and background responses, allowing the facial branches to focus more on gaze-related structural cues. Furthermore, visualization analysis also indicates that the introduction of the coordinate attention module makes the heatmap of the eye branch more concentrated in the corners of the eyes, eyelids, and pupils, further validating its effectiveness in orientation-aware spatial modeling.

[0163] The experimental results demonstrate that the lightweight gaze estimation method based on coordinate attention and facial redundancy compression proposed in this invention achieves accuracies of 3.77°, 5.12°, and 1.55cm (mobile phone) / 2.42cm (tablet) on three mainstream datasets—MPIIFaceGaze, EyeDiap, and GazeCapture—with only 1.54M parameters and 0.54G floating-point operations, respectively. This outperforms most existing methods, achieving an excellent balance between accuracy and efficiency. Ablation experiments validate the effectiveness and complementarity of the coordinate attention module and the facial redundancy compression module: coordinate attention effectively compensates for spatial information loss caused by downsampling in the lightweight network through spatial feature enhancement based on orientation perception; the facial redundancy compression module suppresses redundant responses unrelated to gaze through spatial and channel reconstruction. Visualization analysis further confirms the suppression effect of FRCM on redundant facial responses and the focusing ability of coordinate attention on the eye region. This invention is applicable to real-time gaze estimation in resource-constrained scenarios such as mobile devices, embedded systems, and automotive platforms.

[0164] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression, characterized in that, Includes the following steps: Acquire face image, left eye image, right eye image, and head pose information; The left and right eye images are input into an eye branch with shared weights, and the binocular fusion features are output. The eye branch is constructed based on the MobileNetV4 backbone network and has a coordinate attention module set in its shallow feature stage. Input the face image into a face branch and output the facial features; The facial branch is constructed based on the MobileNetV4 backbone network and has a facial redundancy compression module set in its early downsampling stage. The fusion feature is obtained by fusing the binocular fusion features, facial features, and head posture information; The fused features are input into a regression head to predict the two-dimensional gaze angle.

2. The lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 1, characterized in that, The facial redundancy compression module consists of downsampling convolution, spatial reconstruction unit and channel reconstruction unit connected in series, and is used to suppress facial and background responses that are not related to gaze.

3. The lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 2, characterized in that, The downsampling convolution uses a 3×3 convolution with a stride of 2 to spatially downsample and expand the channels of the input shallow surface features. The spatial reconstruction unit divides the input feature map into several groups along the channel dimension, calculates the similarity between different spatial locations within each group, and filters and merges redundant spatial locations through a threshold to reduce spatial redundancy responses. The channel reconstruction unit divides the input feature map into several groups along the channel dimension, performs intra-group information fusion through a 1×1 convolution, and then weights each channel with channel attention weights generated by global average pooling to suppress redundant channel responses. The output feature map of the facial redundancy compression module is used as the input for subsequent MobileNetV4 layers.

4. The lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 1, characterized in that, The coordinate attention module is used to encode orientation-aware spatial location information along the height and width directions respectively, and to reweight shallow features by generating height direction attention weights and width direction attention weights.

5. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 4, characterized in that, The specific processing steps of the coordinate attention module include: Let the shallow features input to the coordinate attention module be... ,in For the number of channels, For feature map height, The width of the feature map; Perform average pooling along the height direction for each channel to obtain the height direction descriptor. , its first Each component is defined as: ； in , , ; Perform average pooling along the width direction for each channel to obtain the width direction descriptor. , its first Each component is defined as: ； in , ; The height and width descriptors are concatenated along the spatial dimensions, and then sequentially processed through a 1×1 convolution, batch normalization, and ReLU activation function to obtain intermediate features. ; the intermediate features Segmented into two independent directional features along the channel dimension and ;Will and The height-oriented attention weights are generated by passing the weights through a 1×1 convolution and a sigmoid function. Attention weights in the width direction ; Output feature map The Each channel is located in The value at that location is: ； in The first is the height-oriented attention weight. Channel 1 The value of the row, Attention weights in the width direction Channel 1 The value of the column.

6. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 1, characterized in that, The binocular fusion feature is obtained by concatenating the left-eye and right-eye features output from the eye branches into vectors to obtain the concatenated feature. Then, the concatenated feature is input into a fully connected layer and projected to obtain the binocular fusion feature. The facial features are output by the last fully connected layer of the facial branch; The head attitude information is a two-dimensional vector containing the head pitch angle and yaw angle; The fusion feature is a vector obtained by concatenating the binocular fusion feature, facial features, and head pose information.

7. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 1, characterized in that, The regression head is composed of a first fully connected layer and a second fully connected layer connected in series. The input of the first fully connected layer is the fused feature, and the output is activated by the ReLU activation function. The second fully connected layer outputs a two-dimensional gaze angle, which includes a predicted pitch angle and a predicted yaw angle.

8. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 1, characterized in that, The method also Including training steps: Obtain a training sample set, each training sample containing a face image, a left eye image, a right eye image, a two-dimensional head pose vector, and a real two-dimensional gaze angle as a supervision signal; After the training samples undergo feature extraction, fusion, and prediction steps, the predicted two-dimensional gaze angle is obtained. The loss between the predicted 2D gaze angle and the actual 2D gaze angle is calculated using the mean squared error loss function. The network parameters of the eye branch, face branch, and regression head are updated using the backpropagation algorithm. The above process is repeated until the network converges.

9. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 8, characterized in that, During training, data augmentation is performed on the left-eye and right-eye images in the training samples: either the left-eye or right-eye image is randomly selected with a preset probability, and one of the following perturbations is randomly applied to the selected eye image: Random occlusion: Placing a black square at a random location in the image; Gaussian blur: blurs an image using a Gaussian kernel; Brightness adjustment: Multiplies the image brightness by a coefficient; Slight geometric transformation: Randomly rotate by a small angle or translate horizontally / vertically by a few pixels.

10. A lightweight gaze estimation method based on coordinate attention and facial redundancy compression according to claim 1, characterized in that, The MobileNetV4 backbone network adopts the MobileNetV4ConvSmall structure, which includes depthwise separable convolution and a general inverted bottleneck module. The depthwise separable convolution decomposes the standard convolution into channel-wise spatial convolution and pointwise convolution to reduce the number of parameters and computation. The general inverted bottleneck module realizes spatial feature extraction and channel interaction through configurable depthwise convolution.

Images