Long-distance visual positioning method and system for cross-domain scene

Abstract

The invention belongs to the field of non-contact gazing direction positioning and cross-domain self-adaption, and particularly relates to a long-distance visual positioning method and system for a cross-domain scene, and the positioning method comprises the steps: obtaining image data of a target object, the image data comprising face region information and eye region information of the target object; according to the image data, image features in a cross-domain scene are enhanced by using a pre-trained neural network model, and the pre-trained neural network model comprises a feature extraction network composed of multiple layers of convolution residual modules; the feature information obtained after image enhancement processing is input into a regression prediction model, space coordinates of the gazing direction are obtained, the regression prediction model is obtained through training of a training set, and the training set is labeled sample data. According to the invention, long-distance, cross-domain and high-precision eye movement detection can be realized under a non-invasive condition.

Description

Technical Field

[0001] This invention belongs to the field of non-contact gaze orientation positioning and cross-domain adaptation, and particularly relates to a long-distance visual positioning method and system for cross-domain scenarios. Background Technology

[0002] Gazing behavior is a key indicator reflecting human attentional state. Based on this, methods for detecting and analyzing gaze direction have become important research tools in psychology, neuroscience, and human-computer interaction. By capturing and analyzing eye movement trajectories and dynamic characteristics, it is possible to objectively reflect an individual's attention distribution, cognitive activities, and visual information processing patterns in different situations. Based on this technology, scientific evidence and support can be provided for applications such as interface interaction optimization, advertising effectiveness evaluation, driver fatigue monitoring, clinical psychological diagnosis, and educational behavior analysis, thus demonstrating broad application value and practical significance.

[0003] With the rapid development of computer vision and deep learning technologies, a large number of novel eye-tracking detection methods have emerged, including pupil localization algorithms based on image feature points, such as the Starburst algorithm; appearance modeling methods based on deep learning, such as convolutional neural network models on the MPIIGaze and GazeCapture datasets; and algorithms for cross-domain adaptation. However, these methods all rely on high-quality training data and fixed environments, which have significant limitations in practical applications. When there are differences between the training domain and the application domain, such as different lighting conditions, different imaging devices, complex backgrounds, or long-distance imaging conditions, existing methods often fail to accurately extract eye-tracking information, resulting in a significant decrease in detection accuracy in cross-domain scenarios and limiting their widespread use in natural environments.

[0004] At the device level, most existing mainstream eye-tracking detection devices are head-mounted or glasses-based, such as Tobii ProGlasses, EyeLink II, and SMI. These devices typically rely on near-eye infrared light sources and high-frame-rate cameras for eye tracking. While they achieve high accuracy, they have significant drawbacks: first, the wearing process is invasive, interfering with the subject's natural behavior; second, their application scenarios are limited, making widespread deployment in driving, classrooms, or public environments difficult; and third, their high cost hinders promotion in cost-sensitive scenarios such as education and healthcare. Therefore, how to achieve high-precision, long-distance, cross-domain eye-tracking detection without invasiveness has become a pressing issue that needs to be addressed by current technologies. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a long-range visual positioning method and system for cross-domain scenarios. This method can achieve high-precision eye-tracking detection over long distances and across domains under non-invasive conditions.

[0006] To achieve the above objectives, the present invention provides a long-range visual positioning method for cross-domain scenarios, comprising:

[0007] Acquire image data of the target object, wherein the image data includes facial region information and eye region information of the target object;

[0008] Based on the image data, a pre-trained neural network model is used to enhance the image features in cross-domain scenes. The pre-trained neural network model includes a feature extraction network composed of multi-layer convolutional residual modules.

[0009] The feature information obtained after image enhancement is input into the regression prediction model to obtain the spatial coordinates of the gaze direction.

[0010] Optionally, acquiring the image data of the target object includes:

[0011] Image data of a target object is acquired in real time through a visual acquisition device, wherein the visual acquisition device includes at least one variable-focus camera and a computing unit. The variable-focus camera is used to capture human eye movements under different environmental conditions, and the computing unit is used to preprocess and extract features from the image data.

[0012] Optionally, before inputting the image data into the pre-trained neural network model, the image data is preprocessed, the preprocessing including:

[0013] The collected image data is normalized by pixel value normalization to obtain a normalized image;

[0014] A denoised image is obtained by convolving the normalized image with a two-dimensional Gaussian smoothing linear filter.

[0015] A face detection algorithm is used to determine the facial location region in the denoised image and obtain the coordinates of the facial region;

[0016] Based on the facial region coordinates, key image regions including the eyes are extracted and cropped to obtain a standardized input image.

[0017] Optionally, the enhancement of image features in cross-domain scenarios using a pre-trained neural network model includes:

[0018] The preprocessed standardized input image is input into the pre-trained neural network model. The deep features of the low-resolution image are extracted through the multi-layer convolutional residual module. The residual connection structure is used to avoid gradient vanishing and enhance the effective features related to the target. Feature enhancement is achieved in the residual module through multi-level nonlinear mapping. The multi-layer convolutional residual module is composed of several residual modules stacked together.

[0019] The enhanced features are upsampled using a pixel rearrangement operation to output the enhanced image.

[0020] Optionally, obtaining the regression prediction model includes:

[0021] We acquire labeled sample data from the source domain, use ResNet18 as the backbone network for regression prediction, and use the image intensifier modified by EDSR as a supporting module. We pre-train the initial regression prediction model by minimizing the difference between the prediction result and the true gaze direction vector, and optimize the initial parameters of the model.

[0022] After training in the source domain, the initial regression prediction model is transferred to the target domain without the participation of source data. Multiple model snapshots are used to form a prediction ensemble. Multiple forward inferences are performed on the same target domain samples. During the training process in the target domain, the initial regression prediction model is adaptively adjusted through uncertainty reduction strategies, including a variance minimization constraint mechanism and a pseudo-label supervision mechanism.

[0023] After adaptive training, the multiple model parameters obtained from multiple rounds of updates are fused by means to obtain the regression prediction model.

[0024] Optionally, the variance minimization constraint mechanism includes:

[0025] For an enhanced image of a sample from the same target domain, several model snapshots are used to perform forward inference to obtain several predicted gaze vectors.

[0026] Calculate the variance of the plurality of predicted gaze vectors, and use the variance as a quantitative measure of model uncertainty.

[0027] Optionally, the pseudo-label supervision mechanism includes:

[0028] Pseudo-labels are generated using the exponential moving average of the prediction results of the initial regression prediction model on the target domain samples. The pseudo-labels are represented as a weighted average vector in the time dimension.

[0029] A confidence screening mechanism is introduced in the pseudo-label generation process. The stability score of the prediction results is calculated, and only samples with a stability score exceeding a preset threshold are selected as high-confidence samples.

[0030] Based on the pseudo-labels of the high-confidence samples, a supervised loss term is constructed, and the model is iteratively optimized to enable the model to gradually learn the feature distribution of the target domain, thereby making up for the training defects in the absence of source data.

[0031] The present invention also provides a long-range visual positioning system for cross-domain scenarios, comprising: a data acquisition module, an image enhancement module, and a positioning module;

[0032] The data acquisition module is used to acquire image data of the target object, wherein the image data includes facial region information and eye region information of the target object;

[0033] The image enhancement module is used to enhance image features in cross-domain scenes using a pre-trained neural network model based on the image data. The pre-trained neural network model includes a feature extraction network composed of multi-layer convolutional residual modules.

[0034] The positioning module is used to input the feature information obtained after image enhancement processing into the regression prediction model to obtain the spatial coordinates of the gaze direction.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] This invention achieves non-contact positioning at long distances through the coordinated processing of data acquisition, image enhancement, and feature analysis. The method introduces variance constraints and pseudo-label supervision mechanisms to effectively reduce sample and model uncertainties, improve accuracy and stability in cross-domain scenarios, and thus overcome the shortcomings of existing technologies in terms of limited accuracy and high device invasiveness. Attached Figure Description

[0037] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0038] Figure 1 This is a flowchart of a long-range visual positioning method for cross-domain scenarios according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the regression prediction model provided in the embodiments of the present invention;

[0040] Figure 3 This is a flowchart of a long-range visual positioning system for cross-domain scenarios according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention;

[0042] Figure 5 This is a structural diagram of the support device according to an embodiment of the present invention, wherein 500 is the first outer shell and support structure, 501 is the first camera fixing hole, 502 is the first antenna hole, and 503 and 504 are the body fixing holes;

[0043] Figure 6This is another structural diagram of a support device according to an embodiment of the present invention, wherein 600 is the second outer shell and support structure, 601 is the second camera fixing hole, 602 is the SD card fixing hole, 603 is the Type-C power cable fixing hole, and 604 is the second antenna fixing hole. Detailed Implementation

[0044] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0046] This embodiment proposes a long-range visual positioning method for cross-domain scenarios, such as... Figure 1 As shown, the specific steps include:

[0047] Acquire image data of the target object, wherein the image data includes facial region information and eye region information of the target object;

[0048] Based on image data, a pre-trained neural network model is used to enhance image features in cross-domain scenarios. The pre-trained neural network model includes a feature extraction network composed of multi-layer convolutional residual modules.

[0049] The feature information obtained after image enhancement is input into the regression prediction model to obtain the spatial coordinates of the gaze direction.

[0050] Specifically, in this embodiment, the image data of the target object is first acquired in real time through a visual acquisition device. The image data includes information on the face and eye regions of the target object. The visual acquisition device includes at least one zoomable camera and a computing unit. The camera is used to capture human eye movements under different environmental conditions, and the computing unit is used to preprocess the image data and extract features.

[0051] Secondly, based on image data, a pre-trained neural network model is used to enhance and adaptively adjust image features in cross-domain scenarios. The neural network model includes a feature extraction network and a multi-layer convolutional residual module. The feature extraction network is used to extract high-dimensional features related to the target location, and the multi-layer convolutional residual module is used to enhance relevant features and restore details.

[0052] Finally, the feature information obtained after image enhancement is input into the regression prediction model. Variance minimization and pseudo-label supervision mechanisms are introduced during model prediction to further reduce model uncertainty and improve cross-domain adaptive capability. Based on the fusion of cross-domain optimized feature information and contextual information, the multilayer perceptron outputs the spatial coordinates of the gaze direction, achieving high-precision and stable visual localization under long-distance and complex environmental conditions.

[0053] In a preferred embodiment, the pre-trained neural network model is an enhanced deep super-resolution network. This network includes a feature extraction network consisting of initial shallow feature extraction convolutional layers and multiple layers of convolutional residual modules forming the main body of the network.

[0054] Furthermore, acquiring image data of the target object includes:

[0055] The system acquires image data of the target object in real time through a visual acquisition device, which includes at least one variable-focus camera and a computing unit. The variable-focus camera is used to capture human eye movements under different environmental conditions, and the computing unit is used to preprocess the image data and extract features.

[0056] Specifically, the raw image data of the target object is acquired through the camera, including:

[0057] The camera is used to collect image data including the subject's face and eyes. The camera is equipped with an automatic zoom module to automatically adjust the focal length according to the changes in the distance between the subject and the camera.

[0058] According to any implementation of this embodiment, the original image normalization, denoising, and face region cropping include:

[0059] The acquired raw images are normalized for pixel values ​​and then denoised using a Gaussian smoothing linear filter module to reduce the interference of environmental noise and imaging blur on subsequent feature extraction.

[0060] The facial location region is determined by a face detection algorithm, and the facial region is extracted and cropped to ensure accurate processing of subsequent eye movement features.

[0061] Furthermore, before inputting the image data into the pre-trained neural network model, the image data is preprocessed, including:

[0062] The collected image data is normalized by pixel value normalization to obtain a normalized image;

[0063] A denoised image is obtained by convolving the normalized image with a two-dimensional Gaussian smoothing linear filter.

[0064] A face detection algorithm is used to determine the facial location region in the denoised image and obtain the coordinates of the facial region;

[0065] Based on the coordinates of the face region, key image regions including the eyes are extracted and cropped to obtain a standardized input image.

[0066] Specifically, the camera acquires the raw image data of the target object. The raw image may contain resolution differences and noise interference due to imaging distance, lighting changes, or device differences.

[0067] The original image is normalized, denoised, and the face region is cropped to ensure that the input data meets the requirements of subsequent deep network processing.

[0068] The preprocessed image is fed into an enhanced deep super-resolution network, which is used to recover detailed features and improve image quality, thereby reducing sample uncertainty caused by blur and low resolution.

[0069] Perform verification and testing operations on the trained neural network model to ensure that the positioning results of the neural network model meet the preset accuracy requirements in various application environments.

[0070] Specifically, the structure of deep super-resolution networks (such as EDSR) can be divided into three parts:

[0071] 1. Shallow feature extraction module: usually consists of one or several convolutional layers, used to extract preliminary, shallow features from the input low-resolution image.

[0072] 2. Deep Feature Extraction / Enhancement Module: This is the core of the network, composed of multiple stacked "Residual Blocks." This part is responsible for performing complex nonlinear mappings to extract high-frequency details and deep semantic features. The "multi-layer convolutional residual module" mentioned in the claims refers to this part.

[0073] 3. Image Reconstruction / Upsampling Module: Upsamples the enhanced deep feature map to reconstruct a high-resolution image.

[0074] Furthermore, enhancing image features in cross-domain scenarios using pre-trained neural network models includes:

[0075] The preprocessed standardized input image is input into the pre-trained neural network model. Deep features of the low-resolution image are extracted through multi-layer convolutional residual modules. The residual connection structure is used to avoid gradient vanishing and enhance effective features related to the target. Feature enhancement is achieved in the residual module through multi-level nonlinear mapping. The multi-layer convolutional residual module is composed of several residual modules stacked together.

[0076] The enhanced features are upsampled using a pixel rearrangement operation to output the enhanced image.

[0077] Specifically, the preprocessed image is input into an enhanced deep super-resolution network (a pre-trained neural network model) to recover detailed features and improve image quality, including:

[0078] Deep features of low-resolution images are extracted by multi-layer convolutional residual modules in the network, redundant information is removed and effective features related to the target are enhanced;

[0079] Feature enhancement and detail restoration are achieved through multi-level nonlinear mapping in the residual module to reduce blurring and information loss caused by low-resolution acquisition.

[0080] The network's reconstruction module upsamples the enhanced features to output an image with higher resolution and detail.

[0081] Furthermore, obtaining the regression prediction model includes:

[0082] We acquire labeled sample data from the source domain, use ResNet18 as the backbone network for regression prediction, and use the image intensifier modified by EDSR as a supporting module. We pre-train the initial regression prediction model by minimizing the difference between the prediction result and the true gaze direction vector, and optimize the initial parameters of the model.

[0083] After training in the source domain, the initial regression prediction model is transferred to the target domain without the participation of source data. Multiple model snapshots are used to form a prediction ensemble. Multiple forward inferences are performed on the same target domain samples. During the training process in the target domain, the initial regression prediction model is adaptively adjusted through uncertainty reduction strategies, including variance minimization constraint mechanism and pseudo-label supervision mechanism.

[0084] After adaptive training, the multiple model parameters obtained from multiple rounds of updates are fused by means to obtain a regression prediction model.

[0085] Specifically, firstly, based on the labeled sample data in the source domain, the regression prediction model is pre-trained to enable it to learn the basic feature distribution related to the human eye's gaze direction; in this process, the initial parameters of the model are gradually optimized by minimizing the difference between the prediction results and the true labels.

[0086] Secondly, after training in the source domain, the model is transferred to the target domain without the participation of source data, and the model is adaptively adjusted through uncertainty reduction strategies. These include variance constraints based on prediction results to reduce model uncertainty, and a pseudo-label-based supervision mechanism to maintain the model's prediction stability, thereby maintaining high localization accuracy in different scenarios.

[0087] Finally, after adaptive training, the parameters of the model obtained from multiple rounds of updates are fused to obtain a comprehensive regression prediction model that combines stability and accuracy, which can be used for human eye attention localization and visual target tracking under long-distance conditions.

[0088] According to any implementation of this embodiment, the labeled sample data in the source domain includes:

[0089] Image data from publicly available datasets, including at least ETH-XGaze and Gaze360;

[0090] The ETH-XGaze dataset provides a large-scale gaze sample collected under controlled experimental conditions, covering diverse head poses and lighting conditions, which can provide basic gaze direction and feature distribution information for models.

[0091] The Gaze360 dataset contains multi-view samples obtained in natural scenes and complex environments, covering a wider range of gaze angles and environmental changes, thus supplementing the adaptive training of models in cross-domain scenarios.

[0092] Furthermore, the variance minimization constraint mechanism includes:

[0093] For an enhanced image of a sample from the same target domain, several model snapshots are used to perform forward inference to obtain several predicted gaze vectors.

[0094] Calculate the variance of several predicted gaze vectors and use the variance as a quantitative measure of model uncertainty.

[0095] Specifically, if there are significant differences in the prediction results obtained through multiple forward inferences for the same target domain sample, it indicates that the model's output on that sample is uncertain. This embodiment introduces a constraint function that minimizes variance, making the model's prediction results on the target domain data more consistent, thereby effectively reducing model uncertainty.

[0096] The variance minimization constraint is an auxiliary optimization objective in the target domain adaptive training phase. As an uncertainty regularization term, it works in conjunction with the pseudo-label-based main loss function to guide model optimization. These two mechanisms complement each other, jointly improving the model's performance in the unlabeled target domain. Therefore, the variance minimization constraint is a crucial component of the overall optimization objective function, aiming to impose a "consistency" or "determinism" constraint on the model's learning process.

[0097] Furthermore, the pseudo-label monitoring mechanism includes:

[0098] Pseudolabels are generated using the exponential moving average of the prediction results of the initial regression prediction model on the target domain samples. The pseudolabels are represented as a weighted average vector in the time dimension.

[0099] A confidence screening mechanism is introduced in the pseudo-label generation process. The stability score of the prediction results is calculated, and only samples with a stability score exceeding a preset threshold are selected as high-confidence samples.

[0100] The supervised loss term is constructed based on pseudo-labels of high-confidence samples, and the model is iteratively optimized so that the model gradually learns the feature distribution of the target domain, thereby making up for the training defects in the absence of source data.

[0101] By generating pseudo-labels using the model's predictions on target domain samples, adaptive training is achieved when the target domain lacks true labels.

[0102] To avoid the interference of low-confidence predictions on training results, this embodiment introduces a confidence screening mechanism in the pseudo-label generation process, selecting only samples with stable and reliable prediction results as training references.

[0103] By using high-confidence pseudo-labels to iteratively optimize the model, the model can gradually learn the feature distribution of the target domain, thereby making up for the training defects in the absence of source data.

[0104] The positioning method in this embodiment will be described in detail below with reference to the accompanying drawings:

[0105] Step S101: Acquire image data of the subject in real time using a long-distance camera. The image data includes at least facial and eye area information. This module is equipped with an automatic zoom unit to adapt to changes in the distance between the subject and the camera, ensuring clear image input even at long distances. The raw image data acquired by this module provides the basic input for subsequent image enhancement and feature analysis modules.

[0106] In its implementation, the data acquisition section includes a camera module based on the ESP32 series, used to acquire image data of the subject in real time at long distances. The image data includes at least facial and eye area information. The camera module has remote transmission and edge processing capabilities to ensure the real-time performance and integrity of the acquired data.

[0107] In one embodiment, the data acquisition section employs the ESP32-CAM module, which integrates an OV2640 image sensor, supports a maximum resolution of 1600×1200, and has Wi-Fi communication capabilities. This solution features a simple structure and low cost, making it suitable for applications with low accuracy requirements or experimental scenarios. It can capture facial images well within a distance of 0.5-2 meters, providing raw input for subsequent preprocessing and enhancement.

[0108] In another preferred embodiment, the data acquisition section uses an ESP32-S3-EYE camera module, supporting a maximum resolution of 2048×1536, and has built-in AI acceleration functionality. This solution can acquire high-quality facial and eye images within a range of 1–3 meters or even further, significantly improving the feasibility and robustness of long-distance visual positioning.

[0109] Preferably, the data acquisition unit can also be configured with an automatic zoom unit to dynamically adjust the focal length according to changes in the distance between the object being measured and the camera, thereby ensuring that the image remains clear at different acquisition distances. This function can be achieved by connecting an external motorized zoom lens, which can avoid image blurring caused by the movement or posture changes of the target object, further improving the stability of acquisition under long-distance conditions.

[0110] In a preferred embodiment, the autofocus unit employs a motorized lens module based on the OV5640 image sensor. This module supports 5-megapixel resolution and can automatically adjust the focal length via an I²C interface. Compared to traditional fixed-focus modules, this solution can dynamically adjust the focus within a distance of 1–3 meters or even further, thereby ensuring that the image always maintains high clarity and resolution, meeting the needs of long-distance visual positioning.

[0111] In another embodiment, the autofocus unit can be an Arducam series motorized lens module with a focal length range of 16–50mm and supports remote software control. This solution enables fast and precise zoom adjustment in scenarios with complex lighting or frequent target movement, thereby further improving the stability and adaptability of image acquisition.

[0112] Step S102: Based on image data, enhance and adaptively adjust image features in cross-domain scenarios using a pre-trained neural network model. The neural network model includes a feature extraction network and a multi-layer convolutional residual module. The feature extraction network is used to extract high-dimensional features related to the target location, and the multi-layer convolutional residual module is used to enhance relevant features and restore details.

[0113] In the specific implementation process, before using the pre-trained neural network model for feature enhancement and adaptive adjustment, the acquired images need to be preprocessed. Preprocessing includes normalization, noise reduction, face region cropping, and data format and size adjustment to ensure that the input images meet the standardization requirements of network processing and improve the accuracy and stability of feature extraction in cross-domain scenarios.

[0114] In one embodiment, the image preprocessing process includes three stages: normalization, noise reduction, and face region cropping. The goal of this process is to transform the raw image captured by the camera into a more aesthetically pleasing representation of the image. Convert into a standardized image that meets the input requirements of a deep neural network. This ensures the stability and consistency of feature extraction.

[0115] In a preferred embodiment, the acquired raw image pixel values ​​are first normalized. Let the raw image pixel values ​​be... Its range is Using the formula Calculations are performed, in which, The normalized pixel value is a dimensionless real number used to represent the pixel intensity of the image after normalization. Through normalization, all pixel values ​​are mapped to the [0,1] interval, thereby eliminating the dynamic range inconsistency caused by differences in imaging parameters between different devices. Preferably, this embodiment uses an RGB three-channel image as input, and each channel undergoes the above normalization process independently.

[0116] In another embodiment, the normalized image Noise reduction processing is performed to reduce interference from environmental noise and long-distance imaging blur. Preferably, a two-dimensional Gaussian smoothing filter is used, the core function of which is:

[0117] ;

[0118] In the formula, The standard deviation is preferably within the range of values. , This is a two-dimensional Gaussian function used to calculate the filter weight value at the coordinate position, where x and y are spatial coordinate variables representing the pixel position offset with the filter center as the origin. The optimal kernel size for the filter is... or It is used to perform convolution operations on images to obtain smoothed images. This process effectively reduces random noise in the image while preserving edge structure.

[0119] In another embodiment, face detection and region cropping operations are performed on the denoised image. Preferably, the AdaFace algorithm is used to detect face regions in the image. Let the coordinates of the detected face regions be... Then the clipping region R can be represented as:

[0120] ;

[0121] In a preferred embodiment, the cropped area is extended horizontally by 10–20 pixels to each side to ensure complete coverage of eye features.

[0122] The final cropped face image As input data for subsequent deep super-resolution networks.

[0123] In another embodiment, the cropped image data The resolution was uniformly adjusted to a fixed value, preferably 128×128 pixels, and stored in floating-point format (float32) to ensure compatibility with the input requirements of neural networks. This uniformity in size and data format ensures image consistency across devices and domains.

[0124] In the specific implementation process, the image enhancement and feature extraction network incorporates the EDSR network. This network can restore a lower-resolution input image to a higher-resolution version while preserving key details during the enhancement process, thus providing stable feature input for subsequent cross-domain adaptation and regression prediction.

[0125] In a preferred embodiment, the initial resolution of the input image is 128×128, using an RGB three-channel format. The input data is organized as a four-dimensional tensor:

[0126] ;

[0127] In the formula, This indicates the batch size; preferably, the batch size is between 16 and 64. Indicates RGB three channels, Indicates the input resolution. The input image tensor represents the batch of image data input to the neural network model. This is a set of real numbers, indicating that all elements in the tensor are real values. The preferred input data type is 32-bit floating-point.

[0128] In another embodiment, the EDSR network includes multiple convolutional residual modules for extracting deep image features layer by layer and performing nonlinear mapping. Preferably, the number of residual blocks is 32, and each residual block consists of two convolutional layers and a ReLU activation function. Let the input features be... residual block The calculation formula is:

[0129] ;

[0130] In the formula, Represents the convolution weight matrix. For bias terms, This represents the ReLU activation function. Through this residual structure, the network can progressively enhance the detailed features relevant to the target while avoiding gradient vanishing.

[0131] In another preferred embodiment, the EDSR network uses an upsampling module to restore the enhanced features to a higher resolution image. Let the magnification factor be... The output resolution is:

[0132] ;

[0133] In the formula, H in W in Given the height and width of the input image, H out W out To determine the height and width of the output image, in this embodiment, the input 128×128 image is magnified by a factor of 4, resulting in an output resolution of 512×512. This upsampling process employs a pixel shuffle operation to ensure the structural continuity and detail fidelity of the output image.

[0134] In a preferred embodiment, the enhanced image processed by the EDSR network shows a significant improvement in objective evaluation metrics compared to the original input image. To verify the effectiveness of the image enhancement algorithm used in this embodiment, a performance comparison experiment was conducted between the EDSR network and the LapSRN algorithm, and tests were performed using multiple sets of long-distance acquired images. The experimental results are shown in Table 1.

[0135] Table 1

[0136]

[0137] As can be seen from the comparison results in Table 1, although the EDSR model takes slightly longer to run than the LapSRN model, the EDSR model significantly outperforms the LapSRN model in both PSNR and SSIM objective evaluation metrics. Since the gaze-capture task involved in this embodiment has more stringent accuracy requirements, the EDSR model is preferred as the basic structure of the super-resolution algorithm in this embodiment.

[0138] Step S103: Input the feature information obtained after image enhancement into the regression prediction model. Variance minimization and pseudo-label supervision mechanisms are introduced during model prediction to further reduce model uncertainty and improve cross-domain adaptive capability. Based on the fusion of cross-domain optimized feature information and contextual information, the multilayer perceptron outputs the spatial coordinates of the gaze direction, achieving high-precision and stable visual localization under long-distance and complex environmental conditions.

[0139] In its implementation, the long-range visual localization method for cross-domain scenarios proposed in this embodiment further optimizes cross-domain accuracy through adaptive adjustment and training after image enhancement and feature extraction. This process mainly includes four stages: source domain training, target domain transfer, adaptive optimization, and inference prediction. The specific flowchart is attached. Figure 2 As shown.

[0140] In a preferred embodiment, the ETH-XGaze and Gaze360 datasets are used as labeled training data during the source domain training phase. The ETH-XGaze dataset contains 1,830,000 high-resolution gaze images collected in a controlled experimental environment, covering diverse head poses and lighting conditions, and can provide a stable basic gaze feature distribution for the model. The Gaze360 dataset contains 338,000 images collected in natural environments, covering multiple viewpoints and complex lighting variations, and can supplement the model's adaptive training in cross-domain scenarios.

[0141] In another preferred embodiment, ResNet18 is used as the regression prediction backbone network, and an image intensifier modified from EDSR is used as a cooperating module to enhance the input original face image. Source domain training updates the network parameters by minimizing the difference between the predicted result and the actual gaze direction; its optimization objective can be expressed as:

[0142] ;

[0143] In the formula, This is the source domain supervised loss function, used to constrain the model's prediction error on labeled data. For the source domain image, For real-view labeling, For vision enhancement, This is a line-of-sight prediction model.

[0144] After training in the source domain is complete, this embodiment enters the unsupervised adaptive phase in the target domain. Since the target domain data does not contain labeled data and access to the original samples in the source domain is not permitted, an adaptive mechanism under the condition of source data stripping is adopted. For example... Figure 1 As shown in step S202, the unlabeled image of the target domain first obtains high-quality input through the enhancement module, and then inputs it into a multi-path parallel gaze prediction model. Here, we use... Each model snapshot corresponds to the parameter state at different iterations at the end of the source domain training, forming a predictive ensemble.

[0145] In the target domain adaptation phase, this embodiment simultaneously introduces a variance minimization mechanism and a pseudo-label supervision mechanism, see [link to relevant documentation]. Figure 2 Steps S204-S205.

[0146] Variance minimization mechanism: for samples in the same target domain The variance is calculated using the prediction results of different models, and this variance is used as a measure of model uncertainty. The optimization objective is:

[0147] ;

[0148] In the formula, This is a variance minimization constraint loss function, used to reduce the uncertainty of the model in predicting the target domain. Indicates the first Each model predicts the gaze vector for the enhanced image. Let be the sample index in the target dataset, representing the _ ... The predicted gaze vector for each sample. represents the total number of samples in the dataset. By minimizing the variance, the model's predictions on the target domain samples can converge consistently, reducing cross-domain prediction uncertainty.

[0149] Pseudo-label supervision mechanism: To ensure the stability of the model during the adaptation process, this embodiment uses the time average of the prediction results as pseudo-labels and introduces a confidence-weighted strategy during training, selecting only samples with stable predictions for training. The optimization objective can be expressed as:

[0150] ;

[0151] In the formula, This is a pseudo-labeled supervised loss function used to guide the model to learn the feature distribution of the target domain under unlabeled sample conditions. Indicates a pseudo tag. This represents the weighting factor based on the prediction confidence level.

[0152] Joint optimization objective: The final adaptive process integrates two types of loss functions: ,in The adaptive comprehensive loss function is the overall optimization objective during the target domain training phase. The balancing factor is set to 0.01 in this embodiment.

[0153] After adaptive training is completed, the present invention performs mean fusion of the updated multiple model parameters to obtain the final comprehensive regression prediction model.

[0154] In a preferred embodiment, an adaptively trained and optimized regression prediction model is used to calculate the gaze direction and spatial localization result of the target object. The model output is a gaze direction vector or spatial coordinate value, which represents the gaze direction of the measured object in three-dimensional space.

[0155] Preferably, the output is in three-dimensional coordinates. Given in the form of, where This represents the gaze direction vector in the camera coordinate system. Furthermore, this coordinate system can be normalized to obtain a unit vector, which describes the directionality of the gaze without being affected by the gaze distance.

[0156] In another embodiment, the output result can also be in the form of Euler angles. To represent, where Indicates a horizontal angle. It represents the vertical angle so that it can intuitively reflect the direction of gaze in practical applications.

[0157] This embodiment also provides a long-range visual positioning system for cross-domain scenarios, including: a data acquisition module, an image enhancement module, and a positioning module;

[0158] The data acquisition module is used to acquire image data of the target object, including facial region information and eye region information of the target object;

[0159] The image enhancement module is used to enhance image features in cross-domain scenes based on image data using a pre-trained neural network model. The pre-trained neural network model includes a feature extraction network composed of multi-layer convolutional residual modules.

[0160] The localization module is used to input the feature information obtained after image enhancement into the regression prediction model to obtain the spatial coordinates of the gaze direction.

[0161] Specifically, the data acquisition module is used to acquire image data of the subject in real time via a long-distance camera. The image data includes at least information about the facial and eye regions. This module is equipped with an automatic zoom unit to adapt to changes in the distance between the subject and the camera, ensuring clear image input even at long distances. The raw image data acquired by this module provides the basic input for subsequent image enhancement and feature analysis modules.

[0162] The image enhancement module preprocesses and improves the quality of the acquired raw images. Specifically, it performs normalization, noise reduction, and face region cropping on the raw images, then inputs the processed images into an enhanced deep super-resolution network to restore image detail features and improve resolution. This module effectively reduces noise and blur in long-distance imaging, reduces sample uncertainty, and results in clearer and more stable image features.

[0163] The feature analysis module extracts localization-related features and performs regression prediction using a cross-domain adaptive mechanism. The module introduces uncertainty reduction strategies, including variance constraints based on prediction results to reduce model uncertainty, and a pseudo-label-based supervision mechanism to maintain model stability in the target domain. Finally, the module calculates the coordinates of the gaze direction using the regression prediction model, achieving high-precision long-distance visual localization in cross-domain scenarios.

[0164] According to any embodiment of this invention, image feature enhancement and adaptive adjustment include:

[0165] The image preprocessing module is used to normalize, denoise, and crop the face region of the raw images captured by the camera to eliminate interference caused by changes in lighting, differences in equipment, or long-distance imaging, and to ensure that the input data meets the needs of subsequent deep network processing.

[0166] The image enhancement module is used to input the preprocessed image into the enhanced deep super-resolution network, and restore image details and improve resolution through residual learning and feature mapping;

[0167] The results evaluation module is used to perform verification and testing operations on the trained neural network model.

[0168] According to any embodiment of this example, acquiring raw image data of a target object via a camera includes:

[0169] The automatic zoom module is used to automatically adjust the focal length according to the change in the distance between the target object and the camera, so as to maintain image clarity and resolution at different shooting distances;

[0170] The data acquisition module is used to collect raw image data, including the face and eye areas of the target object, in real time through a camera.

[0171] According to any implementation of this embodiment, the original image normalization, denoising, and face region cropping include:

[0172] The normalization processing module is used to normalize the pixel values ​​of the acquired raw images to eliminate uneven brightness and contrast deviations caused by differences in lighting conditions and imaging equipment.

[0173] The image denoising module is used to perform noise suppression processing on the normalized image. It adopts a Gaussian smoothing linear filter denoising method to reduce the impact of environmental interference and imaging blur on subsequent feature extraction.

[0174] The face region extraction module is used to: identify and locate facial regions using face detection algorithms, and crop key image regions containing the eyes within these regions to ensure that subsequent processing focuses on effective features related to eye movements.

[0175] According to any embodiment of this invention, the preprocessed image is input into an enhanced depth super-resolution network to recover detailed features and improve image quality, including:

[0176] The feature extraction module is used to perform deep feature extraction on the input low-resolution image through the convolutional residual structure, remove irrelevant or redundant information, and highlight effective feature information related to target localization.

[0177] The feature enhancement module is used to further process the extracted features in the residual structure using multi-level nonlinear mapping to enhance and restore image details, thereby reducing blurring and information loss caused by long-distance acquisition.

[0178] The image reconstruction module is used to upsample and reconstruct image features after feature enhancement to generate enhanced images with higher resolution and clearer details, so as to ensure the input quality of subsequent cross-domain adaptation and localization analysis.

[0179] According to any implementation of this embodiment, the training process of the regression prediction model includes:

[0180] The pre-training module is used to train the regression prediction model based on labeled sample data in the source domain, enabling the model to learn the basic feature distribution and establish initial prediction capabilities by optimizing parameters.

[0181] Cross-domain transfer module: Used to transfer the model to the target domain after training in the source domain, and continue to adapt under the condition of no source data, so as to ensure that the model can handle image input in different scenarios;

[0182] The uncertainty optimization module is used to introduce variance constraints on prediction results during the target domain training process to reduce model uncertainty, and combined with the pseudo-label supervision mechanism to improve the stability and accuracy of the model under unlabeled sample conditions;

[0183] The parameter fusion module is used to fuse model parameters after multiple rounds of iterative updates to generate a comprehensive regression prediction model that combines stability and accuracy, in order to meet the accuracy requirements of long-distance visual positioning tasks.

[0184] According to any implementation of this embodiment, the model is adaptively adjusted using an uncertainty reduction strategy, including:

[0185] The variance constraint module is used to make multiple predictions on the target domain samples and compare the differences in the results. When the differences are large, a variance minimization constraint is introduced to keep the model's output consistent under the same input, thereby reducing prediction uncertainty.

[0186] The adaptive adjustment module is used to dynamically optimize model parameters under variance constraints, so that the model gradually adapts to the feature distribution of the target domain, thereby improving stability and positioning accuracy in cross-domain scenarios.

[0187] According to any implementation of this embodiment, the pseudo-label-based supervision mechanism to maintain the predictive stability of the model includes:

[0188] The pseudo-label generation module is used to generate pseudo-labels using model prediction results when there are no real labels in the target domain. It also filters unstable or unreliable predictions through a confidence screening mechanism, retaining only high-confidence samples as training references.

[0189] The iterative optimization module is used to continuously train and optimize the model based on the selected high-confidence pseudo-labels, so that the model gradually adapts to the feature distribution of the target domain, thereby improving the prediction stability and accuracy in cross-domain scenarios.

[0190] The following is a detailed description of this embodiment with reference to the accompanying drawings:

[0191] See Figure 3 A long-range visual positioning system for cross-domain scenarios specifically includes the following components:

[0192] The data acquisition module S301 is used to acquire image data of the tested object in real time through a long-distance camera. The image data includes at least facial and eye area information. This module is equipped with an automatic zoom unit to adapt to changes in the distance between the tested object and the camera, thereby ensuring clear image input even at long distances. The raw image data acquired by this module provides the basic input for subsequent image enhancement and feature analysis modules.

[0193] The image enhancement module S302 is used to preprocess and improve the quality of the acquired raw images. Specifically, it performs normalization, noise reduction, and face region cropping on the raw images, and then inputs the processed images into an enhanced deep super-resolution network to restore image detail features and improve resolution. This module effectively reduces noise and blur in long-distance imaging, reduces sample uncertainty, and results in clearer and more stable image features.

[0194] The feature analysis module S303 extracts localization-related features and performs regression prediction using a cross-domain adaptive mechanism. The module introduces uncertainty reduction strategies, including variance constraints based on prediction results to reduce model uncertainty, and a pseudo-label-based supervision mechanism to maintain model stability in the target domain. Finally, the module calculates the coordinates of the gaze direction using the regression prediction model, achieving high-precision long-distance visual localization in cross-domain scenarios.

[0195] The gaze vector coordinate output module S304 is used to output the three-dimensional coordinates. Or Euler angles The output of the predicted gaze vector is represented in a formal way.

[0196] This embodiment proposes an example of an electronic device, such as... Figure 4As shown, the electronic device S401 includes a data input interface S402, a memory S403, a deep learning acceleration module S404, a processor S405, and a vector output module S406. The modules interact and transmit commands via a system bus, thus forming a complete long-range visual positioning hardware platform.

[0197] Specifically, the data input interface S402 is used to receive image data acquired by a long-range camera. The image data includes at least information about the facial and eye regions. This interface can be implemented using USB, serial port, Ethernet, or wireless communication, ensuring compatibility with different types of image acquisition devices. After initial buffering, the data received by the input interface is transmitted to the memory module S403 for storage and scheduling.

[0198] The memory S403 is used to store the operating system, application programs, and computer-executable instructions related to visual positioning, as well as the raw image data, enhanced image data, and training parameters obtained during the data acquisition phase. In a preferred embodiment, the memory can be high-speed flash memory, DDR memory, or a solid-state drive to ensure fast data read / write and high-concurrency processing requirements.

[0199] The deep learning acceleration module S404 is used to accelerate inference and computation of pre-trained neural network models. This module can be a standalone GPU, NPU, or FPGA chip, used to perform complex matrix operations in image enhancement, feature extraction, and regression prediction. In actual operation, the deep learning acceleration module works in conjunction with the processor S405 to significantly shorten network inference time and improve response speed in cross-domain scenarios.

[0200] The processor S405 schedules the tasks of each module and executes the core algorithm logic. Specifically, it calls the computer program stored in memory to preprocess the acquired images, perform feature analysis, and conduct cross-domain adaptive training, ultimately outputting a prediction result of the human eye's gaze direction vector. The processor can be either a general-purpose CPU or a dedicated embedded controller; this embodiment does not impose any limitations on either. When the processor is combined with the deep learning acceleration module S404, real-time gaze tracking under long-distance conditions can be achieved.

[0201] Finally, the vector output module S506 outputs the human eye gaze vector obtained through regression prediction. The output format can be either three-dimensional coordinates. It can also be Euler angles. This allows the module to meet the needs of different application scenarios. The output module can transmit the results to user terminals, servers, or external business systems via a communication interface for tasks such as driver status monitoring and human-machine interaction control.

[0202] Furthermore, the electronic device can maintain a connection with a remote server via a communication module to achieve data interaction and model updates. The server can be a single computer device or a distributed cluster of multiple devices, used for large-scale model training and parameter scheduling in cross-domain scenarios. Through the above design, the device in this embodiment ensures high accuracy while also possessing good scalability and environmental adaptability.

[0203] In one embodiment, it should be noted that, in addition to the core hardware components described above, this embodiment also proposes a shell and support structure based on 3D printing technology, used to fix and protect the electronic device and provide assembly support. See also Figure 5 The first outer casing and support structure 500 includes: a first camera mounting hole 501, a first antenna mounting hole 502, and body mounting holes 503 and 504, as well as other necessary components. The structure and function of each component are described below:

[0204] 501: First camera fixing hole, used for axial and radial positioning and limiting of the camera module, so that the camera optical axis is coaxial with the body reference surface / reference hole, ensuring that the imaging center is concentric with the opening; and as the mounting interface of the optical window, it can be used with transparent protective parts and sealing rings to achieve dustproof / waterproof and impact-resistant protection.

[0205] 502: First antenna aperture, used as an RF transparent window to reduce dielectric loss and metal obstruction affecting antenna transmission / reception, and improve radiation efficiency and directionality. Figure 1 To the point of being compatible;

[0206] 503 and 504: Body mounting holes for housing magnets that attract the base magnets, thereby detachably fixing the body structure to the base for quick assembly and maintenance;

[0207] In another embodiment, this embodiment also proposes a different shell and support structure based on 3D printing technology for fixing, protecting, and providing assembly support for electronic devices. See also Figure 6 The second housing and support structure 600 includes: a second camera mounting hole 601, an SD card mounting hole 602, a Type-C power cable mounting hole 603, a second antenna mounting hole 604, and other necessary components. The structure and function of each component are described below:

[0208] 601: Second camera mounting hole, used for axial and radial positioning and limiting of the camera module, so that the camera optical axis is coaxial with the body reference surface / reference hole, ensuring that the imaging center is concentric with the opening; and as the mounting interface of the optical window, it can be used with transparent protective parts and sealing rings to achieve dustproof / waterproof and impact-resistant protection.

[0209] 602: SD card fixing hole, used for precise positioning and fixing of SD / microSD card slots or card trays, ensuring the limit of insertion and removal direction and depth; provides an interface for external operation window or opening structure, facilitating card replacement and maintenance in the packaged state of the whole device;

[0210] 603: Type-C power cable mounting hole, used as an entry and fixing channel for USB Type-C power cables, or as a panel mounting position for Type-C female connectors / adapter components, to achieve mechanical stability and position reference for the power interface;

[0211] 604: Second antenna mounting hole, used as an RF transparent window to reduce dielectric loss and metal obstruction affecting antenna transmission / reception, and improve radiation efficiency and directionality. Figure 1 To the point of being responsive.

[0212] Furthermore, the outer shell and supporting structure S500 are integrally formed using a 3D printing process, preferably using a polymer material with good dimensional stability and weather resistance, such as PLA or PETG. Key mating holes can be post-processed and surface-treated to improve dimensional accuracy, sealing, and assembly consistency. The shape, size, number, and specific location of the aforementioned holes can be optimized and adjusted according to the shape of the electronic device, optical / RF performance, and assembly process, and are not limited to the illustrated embodiment.

[0213] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A long-range visual positioning method for cross-domain scenarios, characterized in that, include: Acquire image data of the target object, wherein the image data includes facial region information and eye region information of the target object; Based on the image data, a pre-trained neural network model is used to enhance the image features in cross-domain scenes. The pre-trained neural network model includes a feature extraction network composed of multi-layer convolutional residual modules. The feature information obtained after image enhancement is input into the regression prediction model to obtain the spatial coordinates of the gaze direction. The regression prediction model is obtained by training a training set, which is labeled sample data.

2. The long-range visual positioning method for cross-domain scenarios according to claim 1, characterized in that, Obtaining the image data of the target object includes: Image data of a target object is acquired in real time through a visual acquisition device, wherein the visual acquisition device includes at least one variable-focus camera and a computing unit. The variable-focus camera is used to capture human eye movements under different environmental conditions, and the computing unit is used to preprocess and extract features from the image data.

3. The long-range visual positioning method for cross-domain scenarios according to claim 1, characterized in that, Before inputting the image data into the pre-trained neural network model, the image data is preprocessed, the preprocessing including: The collected image data is normalized by pixel value normalization to obtain a normalized image; A denoised image is obtained by convolving the normalized image with a two-dimensional Gaussian smoothing linear filter. A face detection algorithm is used to determine the facial location region in the denoised image and obtain the coordinates of the facial region; Based on the facial region coordinates, key image regions including the eyes are extracted and cropped to obtain a standardized input image.

4. The long-range visual positioning method for cross-domain scenarios according to claim 3, characterized in that, Enhancing image features in cross-domain scenarios using pre-trained neural network models includes: The preprocessed standardized input image is input into the pre-trained neural network model. The deep features of the low-resolution image are extracted through the multi-layer convolutional residual module. The residual connection structure is used to avoid gradient vanishing and enhance the effective features related to the target. Feature enhancement is achieved in the residual module through multi-level nonlinear mapping. The multi-layer convolutional residual module is composed of several residual modules stacked together. The enhanced features are upsampled using a pixel rearrangement operation to output the enhanced image.

5. The long-range visual positioning method for cross-domain scenarios according to claim 1, characterized in that, Obtaining the regression prediction model includes: We acquire labeled sample data from the source domain, use ResNet18 as the backbone network for regression prediction, and use the image intensifier modified by EDSR as a supporting module. We pre-train the initial regression prediction model by minimizing the difference between the prediction result and the true gaze direction vector, and optimize the initial parameters of the model. After training in the source domain, the initial regression prediction model is transferred to the target domain without the participation of source data. Multiple model snapshots are used to form a prediction ensemble. Multiple forward inferences are performed on the same target domain samples. During the training process in the target domain, the initial regression prediction model is adaptively adjusted through uncertainty reduction strategies, including a variance minimization constraint mechanism and a pseudo-label supervision mechanism. After adaptive training, the multiple model parameters obtained from multiple rounds of updates are fused by means to obtain the regression prediction model.

6. A long-range visual positioning method for cross-domain scenarios according to claim 5, characterized in that, The variance minimization constraint mechanism includes: For an enhanced image of a sample from the same target domain, several model snapshots are used to perform forward inference to obtain several predicted gaze vectors. Calculate the variance of the plurality of predicted gaze vectors, and use the variance as a quantitative measure of model uncertainty.

7. A long-range visual positioning method for cross-domain scenarios according to claim 5, characterized in that, The pseudo-label monitoring mechanism includes: Pseudo-labels are generated using the exponential moving average of the prediction results of the initial regression prediction model on the target domain samples. The pseudo-labels are represented as a weighted average vector in the time dimension. A confidence screening mechanism is introduced in the pseudo-label generation process. The stability score of the prediction results is calculated, and only samples with a stability score exceeding a preset threshold are selected as high-confidence samples. Based on the pseudo-labels of the high-confidence samples, a supervised loss term is constructed, and the model is iteratively optimized to enable the model to gradually learn the feature distribution of the target domain, thereby making up for the training defects in the absence of source data.

8. A long-range visual positioning system for cross-domain scenarios implemented by the method according to any one of claims 1-7, characterized in that, include: Data acquisition module, image enhancement module, and positioning module; The data acquisition module is used to acquire image data of the target object, wherein the image data includes facial region information and eye region information of the target object; The image enhancement module is used to enhance image features in cross-domain scenes using a pre-trained neural network model based on the image data. The pre-trained neural network model includes a feature extraction network composed of multi-layer convolutional residual modules. The positioning module is used to input the feature information obtained after image enhancement processing into the regression prediction model to obtain the spatial coordinates of the gaze direction.

Images