Fusion-based adaptive pupil interaction method

Abstract

The application discloses a fusion-based adaptive pupil interaction method and belongs to the technical field of pupil positioning, and the steps of the method comprise the following steps: HOG feature extraction cooperates with an SVM model to predict key points, lock the iris position and calculate the centroid; based on the screen ratio of a display, a spatial calibration model with the same ratio as the display screen is constructed, interpolation algorithms are used to optimize pupil data, the corrected coordinate data is projected to the display, the actual position of the line-of-sight point in the screen is determined, the parameter vector of the model is iteratively updated through an L-BFGS-B optimization algorithm, the coordinate data is iteratively optimized through an HBG algorithm after mapping, and the fixation center point is determined. The application has the beneficial effects that: in view of the uncertainty of the pupil position, a calibration model is constructed by using device parameters, the data is finely calibrated by using interpolation and optimization algorithms, the error is greatly reduced, the prediction deviation is further reduced through the iterative optimization of the HBG algorithm, and the system has the advantages of simple structure, high lightweight degree, high positioning accuracy and fast response speed.

Description

Technical Field

[0001] This invention relates to the field of pupil localization technology, and in particular to an adaptive pupil interaction method based on fusion. Background Technology

[0002] Currently, numerous scholars have conducted in-depth research on pupil image features in different application scenarios and have extensively discussed pupil localization techniques. For example, a hierarchical pupil localization method for helmet-mounted eye-tracking systems achieves a balance between speed and accuracy, but fails to detect the specific target location of the gaze. Based on pupil image features from video nystagmus, a method first performs coarse iris localization and then precise pupil localization; this algorithm exhibits good anti-interference and accuracy, but suffers from slow speed and is easily affected by environmental factors. Infrared methods, as a non-invasive eye-tracking technology, do not cause discomfort to the eyes and have high accuracy, but they rely on external devices and are easily interfered with in multi-light source environments.

[0003] Current gaze-based display technologies largely suffer from problems such as high susceptibility to environmental influences, low accuracy, numerous limitations on recognition and reasoning (e.g., the relative position of the device and the eyes), complex system structures, inability to be lightweight, and poor compatibility. Several smart glasses currently on the market primarily utilize VR and AR technologies, but rely on peripherals like eye trackers, resulting in bulky devices unsuitable for everyday wear and prohibitively high manufacturing costs. Other smart glasses simply incorporate some functional modules (such as translation or bundled applications), but application selection still relies on peripherals like controllers. These products are more suitable for entertainment than frequent use during daily commutes. Summary of the Invention

[0004] The purpose of this invention is to provide a fusion-based adaptive pupil interaction method to solve the problems of current pupil positioning technology in practical applications, such as being too sensitive to environmental changes, slow response speed, strong dependence on the relative position of the device and the eye, complex system operation structure, insufficient lightweightness, and poor compatibility.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] The steps of the fusion-based adaptive pupil interaction method include:

[0007] Pupil recognition and data collection: The input facial image is divided into multiple small regions by HOG feature extraction. The gradients of pixels in the horizontal and vertical directions are calculated to obtain the gradient magnitude and direction. The SVM model is used to classify each small region, predict the location of key points, create an eye mask and extract the iris region. Thresholding is applied to determine the iris position. Then, the area of ​​the iris region and its weighted sum on the horizontal and vertical coordinates are calculated by the zeroth moment and the first moment to obtain the centroid coordinates and determine the pupil position to achieve the original data collection.

[0008] Adaptive calibration model and data mapping: Based on the screen ratio of the display, a spatial calibration model with the same ratio as the display screen is constructed using the collected data. The spatial calibration model uses an interpolation algorithm to optimize the collected pupil data, correct the coordinates of unknown points, and the corrected coordinate data is mirrored and adapted and projected onto the display to determine the actual position of the gaze point on the screen.

[0009] Data calibration and optimization: The spatial calibration model uses the L-BFGS-B optimization algorithm to iteratively update the parameter vector of the model. After the coordinate data is mapped, the error is iteratively optimized using the HBG algorithm to determine the gaze concentration point.

[0010] Furthermore, the interpolation algorithm optimizes the collected pupil data and corrects the coordinates of unknown points by: using the collected pupil position data as a basis, constructing an interpolation model on a regular grid using interpolation methods, generating interpolation results on the regular grid and standardizing them, and calibrating the identified pupil position to a standard position.

[0011] Furthermore, the process of mirroring and projecting the corrected coordinate data onto the display includes: performing a horizontal mirror transformation on the corrected coordinate data; projecting the mirrored coordinate data onto the display according to the mapping relationship between the spatial calibration model and the display; and then reversing the mirror transformation to determine the actual position of the viewing point on the screen.

[0012] Furthermore, the L-BFGS-B optimization algorithm adds upper and lower bound constraints on the parameters, and its iterative update formula is as follows: in, It is the parameter vector of the k-th iteration. It's the step length. It is an approximate inverse Hessian matrix. It is the current parameter The gradient at that point.

[0013] Furthermore, the X and Y axes of the spatial calibration model correspond to the aspect ratio of the display screen, while the Z axis of the spatial calibration model represents the dynamic deviation.

[0014] Furthermore, the HBG algorithm guides the model optimization direction by calculating the negative gradient of the loss function, uses a base learner to fit the negative gradient, and then determines the optimal step size to update the model through line search.

[0015] The present invention has the following advantages:

[0016] By using HOG feature extraction and collaborating with an SVM model to predict key points, the system accurately locates the iris and calculates its centroid. To address the uncertainty in pupil position, a spatial scaling calibration model is constructed using device display parameters. Through interpolation and optimization algorithms, the data is finely calibrated, significantly reducing errors. Further optimization using the HBG algorithm further reduces prediction bias. The system features a simple structure, high lightweight design, high positioning accuracy, and fast response speed. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of data calibration and projection.

[0018] Figure 2 This is the logical operation diagram of the system.

[0019] Figure 3 This is a schematic diagram of the predicted X value on a regular grid.

[0020] Figure 4 This is a schematic diagram of the predicted Y value on a regular grid.

[0021] Figure 5 A flowchart illustrating the usage process of an interactive integration system.

[0022] Figure 6 This is a diagram showing the location of key facial features.

[0023] Figure 7 This is a schematic diagram of the center point of the iris region.

[0024] Figure 8 This is an error distribution diagram.

[0025] Figure 9 This is a schematic diagram of the residual distribution results.

[0026] Figure 10 This is a schematic diagram of the HBG algorithm flow.

[0027] Figure 11 This is a flowchart of the interpolation algorithm and optimization process. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0029] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this invention can be combined with each other.

[0031] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0032] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention. In addition, the terms "first," "second," etc., are only used to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0033] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0034] refer to Figure 1-11 As shown, one embodiment of the present invention is as follows:

[0035] The steps of the fusion-based adaptive pupil interaction method include:

[0036] Pupil Recognition and Data Collection: A pupil recognition system is built using OpenCV technology. Facial images of the user are captured by a camera and input as raw data to an OpenCV-based image processing module. The OpenCV image processing module uses HOG feature extraction to divide the input facial image into multiple 8×8 pixel regions. For each pixel (x, y), its horizontal and vertical gradients are calculated to obtain the gradient magnitude and direction. The specific formulas are as follows:

[0037] Horizontal gradient:

[0038] Vertical gradient:

[0039] Pixel value of a certain point:

[0040] Calculate the gradient magnitude and direction:

[0041]

[0042] The trained SVM model is used to classify each small region and predict the locations of 68 key points in the image, such as... Figure 6 As shown in the figure, the area marked by feature points 36-47 is the eye region. The horizontal axis represents the order of the feature points arranged horizontally on the face, and the vertical axis represents the vertical pixel coordinates of the point on the computer screen. Based on the keypoint localization, an eye mask is created, filling the eye region with gray. The iris region is extracted from the mask, and thresholding is applied to determine the iris position. It can be understood that the fill color of the eye region can also be white; those skilled in the art can implement this according to the specific circumstances. Assuming the input feature vector is x, the output feature point positions are:

[0043] In the formula, x is the input feature vector, representing the original feature representation extracted from the image. The output feature point location is the set of coordinates of key points such as the eye and iris predicted by the model, or a vector representing these locations. θ is the model parameter: it is a learnable (or pre-set) parameter inside the model f, which determines the mapping relationship from input to output.

[0044] Subsequently, the area of ​​the iris region and its weighted sum on the horizontal and vertical axes are calculated using the zeroth and first moments to obtain the centroid coordinates, thus determining the pupil position and completing the raw data acquisition. Figure 7 As shown. Wherein:

[0045] Zeroth moment:

[0046] First moment in the transverse direction:

[0047] Longitudinal first moment:

[0048] Centroid calculation:

[0049] In the formula, To measure the total area of ​​the iris region, To measure the weighted sum of the horizontal coordinates of the iris region across all pixels, To measure the weighted sum of the ordinates of the iris region across all pixels, These are the coordinates of the centroid. For example... Figure 2 The logic diagram shown illustrates the process from image acquisition to pupil localization. The iris is detected based on color information (red channel) and morphological operations, and the pupil position is determined by the centroid, thus acquiring the raw data. This data is then combined with the device's display parameters to construct a spatial scaling calibration model.

[0050] Adaptive Calibration Model: Addressing the uncertainty of pupil position, this invention utilizes device display parameters to construct a spatial scaling calibration model and employs a three-dimensional mathematical model to describe the deviation between the collected data and the calibration data. Specifically, this embodiment, based on the display's screen ratio, uses the collected data to construct a spatial calibration model with the same ratio as the display screen. This spatial calibration model uses an interpolation algorithm to optimize the collected pupil data and correct the coordinates of unknown points.

[0051] like Figure 1 As shown, specifically, the X and Y axes of the spatial calibration model correspond to the aspect ratio of the display screen to ensure the geometric correspondence between the two-dimensional plane and the screen pixels. The Z-axis of the spatial calibration model represents the dynamic deviation, mapping the actual measured deviation values ​​(such as position offset and deformation error) to Z-axis coordinates to form a spatial height field, which intuitively reflects the error distribution in different areas.

[0052] In model construction, the center position p(hex0, hey0) of the calibration model is determined by calculating the centroid positions of the left and right pupils at the center point of the screen twice (Left Eye X, Left Eye Y) and (Right Eye X, Right Eye Y), and then analyzing the gaze points collected at the left and right edges of the calibration model, and determining the relative proportional length of the calibration template in combination with the screen ratio. The specific formula is as follows:

[0053] Determine the center point P of the calibration model:

[0054]

[0055]

[0056] Determine the relative proportion length (long):

[0057] In the formula, x1 is the coordinate of the known calibration point on the right side of the calibration model, and x2 is the coordinate of the known calibration point on the left side of the calibration model.

[0058] In practical implementation, the origin and coordinate system of the spatial calibration model can be flexibly defined. The origin can be the center of the screen or a user-specified reference point, and the coordinate system can be adapted to different installation scenarios, such as multi-screen splicing and curved screens.

[0059] The model scale is determined by the relative proportions of the known calibration points and the monitor screen, forming the basic framework of a spatial calibration model identical to the screen. Due to dynamic deviations, the model's rule network exhibits varying degrees of height. Therefore, the model employs an interpolation algorithm to calculate these dynamic deviations, optimize the coordinate data, and correct the coordinates of unknown points.

[0060] Based on the collected pupil position data, the relative coordinates of the line-of-sight point in the calibration template relative to the calibration center are calculated in the calibration model. Preliminary position association is achieved by matching and aligning with the calibration model. Interpolation methods are then used on a regular grid (…). , Construct an interpolation model to generate interpolation results on a regular grid. , Then, the coordinate data is standardized and projected after correction. Corresponding to The deviation value, Corresponding to The deviation value, such as Figure 3 As shown in Figure 4.

[0061] The identified pupil position is calibrated to a standard position using an interpolation function to dynamically correct for errors at unknown points. The specific formula is as follows:

[0062]

[0063]

[0064] Where f is the interpolation function, μ is the mean, σ is the standard deviation, and X scaled Here, X represents the standardized data values, where X is a single data point in the original dataset. Let be the input independent variable corresponding to the i-th sample, representing the spatial coordinates. The output result is obtained after the i-th sample (or the i-th group of data) is calculated by the interpolation function f.

[0065] In terms of data acquisition and updating of the spatial calibration model, due to environmental changes or hardware aging, the calibration model needs to be adaptively adjusted. This can be achieved by continuously acquiring screen display data (such as marker point positions and edge distortion) through sensors or cameras, dynamically correcting model parameters to adapt to environmental changes (temperature, humidity) or hardware aging. Higher resolution Z-axis sampling points are allocated to distorted or physically damaged areas at the screen edges to capture subtle deviations. In addition to geometric deviations, the Z-axis can be expanded to a comprehensive error index (such as a weighted value for color cast and brightness unevenness) to achieve multi-parameter joint calibration. Furthermore, error change models (such as thermal expansion effects) can be established based on historical data to predict and correct deviation trends in advance, reducing real-time calibration latency.

[0066] Data Mapping: The corrected coordinate data is projected onto the display according to the mapping relationship between the spatial calibration model and the display, determining the actual position of the gaze point on the screen. Since image mirroring may occur during actual data acquisition (e.g., differences in orientation between the acquired image and the screen display), this embodiment performs mirroring processing on the data mapping. The corrected coordinate data is horizontally mirrored, and then the relative coordinates are projected onto the display according to the mapping relationship between the spatial calibration model and the display. After completing the screen projection calculation, the mirror transformation is reversed to determine the actual position of the gaze point on the screen. Through a closed-loop processing mode of "calibration model conversion → mirror adaptation → projection → restoration," the directional deviation between acquisition and display is eliminated, improving the positioning accuracy of the pupil gaze point on the screen.

[0067] More specifically, the spatial calibration model incorporates the L-BFGS-B optimization algorithm to iteratively update the parameter vector, continuously optimizing the model parameters until an optimal state is reached. This results in more accurate pupil position predictions. The introduction of the L-BFGS-B optimization algorithm improves data accuracy and reliability, providing high-quality data support for subsequent interactive operations. Figure 11 As shown.

[0068] L-BFGS (Limited Memory BFGS algorithm) is a quasi-Newton optimization algorithm that accelerates iteration by approximating the inverse Hessian matrix. This embodiment uses an extended version of L-BFGS-B, which adds upper and lower bound constraints on the parameters to better adapt to optimization scenarios with limited parameter value ranges (such as the constraints on model parameters in this embodiment) and address parameter boundary limitations. The iterative update formula for the L-BFGS-B optimization algorithm is as follows:

[0069]

[0070] In the formula, θ k Let be the parameter vector for the k-th iteration; The step size is determined through line search; It is an approximate inverse Hessian matrix; For the current parameter The gradient at that point.

[0071] Choose initial parameters to set the initial approximate inverse matrix. Calculate the current parameters. gradient at Using an approximate inverse matrix Calculate search direction The step size is determined by line search. Make The constraints are met. Update parameters. Check for convergence: If the gradient norm is... (where 'a' is a preset threshold, which needs to be determined based on the scenario, such as pupil prediction scenarios. Due to the balance between data accuracy and computational cost, it is often set to the order of (10^-4 to 10^-6)). If 'a' is a preset threshold, the iteration stops; otherwise, continue to the next step. Update the approximate inverse Hessian matrix: use the latest s k and y k Update H k Get H k+1 Return to the next iteration.

[0072] L-BFGS updates the approximate inverse Hessian matrix in the following way:

[0073]

[0074]

[0075]

[0076]

[0077] In the formula, I is the identity matrix, ρ k H is a scalar k Let θ be the approximate inverse matrix of the k-th iteration. k Let θ be the parameter vector for the k-th iteration. k+1 Let S be the parameter vector for the (k+1)th iteration. k For the parameter update amount, y k The change in the gradient of the objective function. : is the gradient change y k The transpose of y, due to gradient y k It is often used in calculations as a column vector, and after transpose it becomes a row vector to satisfy the dimension matching of matrix multiplication.

[0078] First, in each iteration, S is calculated. K and y kThese reflect the shift in parameters and the change in the gradient of the objective function, respectively, and are used for subsequent updates of H. k The key input. Next, based on S... k and y k Calculate the scalar ρ k It is used to scale the relevant terms involved in subsequent update calculations to ensure the rationality and effectiveness of the values ​​in the update process.

[0079] Then, using the constructed I and S K y k ρ k And the current H k The formula is used to update and obtain H for the next step (step k+1). k+1 This iterative update process allows H to... k It can approximate the properties of the true inverse Hessian matrix well, thus providing support for the parameter update based on Hessian matrix information in quasi-Newton algorithms (such as determining a better search direction), and helping optimization algorithms to find the optimal solution of the objective function more efficiently.

[0080] By analyzing and modeling known points using interpolation algorithms and L-BFGS-B optimization algorithms, the error values ​​of unknown points are predicted, thereby achieving accurate calibration of the gaze point position.

[0081] After coordinate data mapping, the Hit Gradient Boosting (HBG) algorithm is used to iteratively optimize the error, determine the gaze concentration point, and further improve positioning accuracy, such as... Figure 10 As shown. Using the HBG algorithm and the gradient descent approach, the negative gradient of the loss function of the current model is calculated in each iteration. A weak learner is obtained by fitting the negative gradient using a base learner. Then, the optimal step size is determined through line search to update the model, ultimately constructing a high-quality model. The specific steps are as follows:

[0082] Initialize the model, initialize the model It is a constant, typically representing all target values. Average value:

[0083]

[0084] Calculate the negative gradient of the current model on all samples. For the m-th iteration, calculate the current... At each sample point x i The negative gradient at point L, where L is the loss function:

[0085]

[0086] Use base learners (such as regression trees) to fit negative gradients. To obtain a weak learner :

[0087]

[0088] Use line search to determine the optimal step size ρ m :

[0089]

[0090] After repeating the above steps m times, the final model is obtained through updating. :

[0091]

[0092] In the formula, For the cumulative model in the m-th iteration, For the (m-1)th iteration of the cumulative model, ρ m The weighting factor is also known as the learning rate. Let γ be the m-th weak learner, and γ be the true label (target value) of the training sample. The gradient direction for calculating the error between the current model prediction and the true label is the standard format for "differentiation with respect to a variable" in calculus, used to specify the object of differentiation. For the learner in sample x i The output is given by h, which represents the algorithm for constructing the weak learner. For the m-th regression tree in sample x i The output is shown above. In gradient boosting, the model is iteratively optimized step by step through "calculating the error gradient → fitting the gradient with the base learner → updating the model" to make the final value closer to the true label and improve the localization accuracy.

[0093] After initializing the model parameters, the negative gradient of the loss function is calculated to guide the model optimization direction. A regression tree is used to fit the negative gradient of the current iteration, and the model is updated accordingly. If the preset iteration termination condition is not met, the next round of optimization continues; otherwise, the optimized final model is output. In each iteration, a line search method is used to determine the optimal step size, and the model is updated accordingly. This process is repeated until the model converges, resulting in a final model with better performance and higher prediction accuracy, effectively improving its ability to process complex pupil interaction data. A neural network interpolation algorithm is used to quickly and accurately predict unknown point errors, combined with the Hit Gradient Boosting algorithm to optimize the overall system error for dynamic calibration and error optimization of pupil position, improving positioning accuracy.

[0094] This invention utilizes an image processing-based pupil recognition method as the foundational operation of a pupil interaction system. In terms of interaction integration, it integrates this method with modules such as intelligent voice and motion-sensing interaction to construct a comprehensive interactive system, such as... Figure 5 As shown, this system provides a low-latency, high-precision, and portable pupil-based operating system for head-mounted devices. The system employs a B / S architecture for user interaction, with a front-end mobile application handling user interaction and data collection and display. The back-end server handles data processing, image recognition, and business logic execution, while a database stores and manages the relevant data. Users initiate interaction requests through the front-end interface, and the back-end server calls the corresponding modules to complete tasks such as pupil recognition, data calibration, and optimization calculations. After processing, the results are returned to the front-end, enabling real-time interaction between the user and the system. The system supports functions such as controlling the mouse via pupils, locking text content, and obtaining voice feedback, effectively improving the user experience and interaction efficiency.

[0095] This invention focuses on lightweighting human eye gaze display technology. By introducing an intelligent module and constructing a fusion-based adaptive pupil interaction algorithm, it uses a camera to accurately identify the relative position of the pupil on the screen and further predicts the position of the human eye's gaze point. This enables efficient target recognition, selection, and operation functions. The mathematical calculations used can solve the problem of lightweighting existing interactive systems. It has good real-time performance, a simple operating structure, small memory footprint, fast operation speed, and strong compatibility.

[0096] By using HOG feature extraction in conjunction with an SVM model to predict keypoints, and combining centroid calculation and thresholding, the iris position is accurately located and the centroid is calculated, laying a solid foundation for gaze point prediction. An adaptive spatial calibration model is constructed based on screen ratio and collected data, and the data is finely calibrated using interpolation and optimization algorithms, significantly reducing errors. Further optimization using the Hit Gradient Boosting algorithm further reduces prediction bias. It performs excellently on both the training and test sets. Figure 8 The error distribution diagram shown and Figure 9 The residual distribution results shown indicate that the present invention has low error values ​​and high coefficient of determination.

[0097] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fusion-based adaptive pupil interaction method, characterized in that: The steps include: Pupil recognition and data collection: The input facial image is divided into multiple small regions by HOG feature extraction. The gradients of pixels in the horizontal and vertical directions are calculated to obtain the gradient magnitude and direction. The SVM model is used to classify each small region, predict the location of key points, create an eye mask and extract the iris region. Thresholding is applied to determine the iris position. Then, the area of ​​the iris region and its weighted sum on the horizontal and vertical coordinates are calculated by the zeroth moment and the first moment to obtain the centroid coordinates and determine the pupil position to achieve the original data collection. Adaptive calibration model and data mapping: Based on the screen ratio of the display, a spatial calibration model with the same ratio as the display screen is constructed using the collected data. The spatial calibration model uses an interpolation algorithm to optimize the collected pupil data, correct the coordinates of unknown points, and the corrected coordinate data is mirrored and adapted and projected onto the display to determine the actual position of the gaze point on the screen. Data calibration and optimization: The spatial calibration model uses the L-BFGS-B optimization algorithm to iteratively update the parameter vector of the model. After the coordinate data is mapped, the error is iteratively optimized using the HBG algorithm to determine the gaze concentration point. The process of mirroring and adapting the corrected coordinate data and projecting it onto the display includes: performing a horizontal mirror transformation on the corrected coordinate data; projecting the mirrored coordinate data onto the display according to the mapping relationship between the spatial calibration model and the display; and then reversing the mirror transformation to determine the actual position of the viewing point on the screen. The L-BFGS-B optimization algorithm adds upper and lower bound constraints on the parameters, and its iterative update formula is as follows: in, It is the parameter vector of the k-th iteration. It's the step length. It is an approximately inverse Hessian matrix, ∇f( ) is the current parameter gradient at; The X and Y axes of the spatial calibration model correspond to the aspect ratio of the display screen, and the Z axis of the spatial calibration model represents the dynamic deviation. The actual measured deviation value is mapped to the Z-axis coordinate to form a spatial height field.

2. The fusion-based adaptive pupil interaction method according to claim 1, characterized in that: The interpolation algorithm optimizes the collected pupil data and corrects the coordinates of unknown points by: using the collected pupil position data as a basis, constructing an interpolation model on a regular grid using an interpolation method, generating interpolation results on the regular grid and standardizing them, and calibrating the identified pupil position to a standard position.

3. The fusion-based adaptive pupil interaction method according to claim 1, characterized in that: The HBG algorithm guides the model optimization direction by calculating the negative gradient of the loss function, uses a base learner to fit the negative gradient, and then determines the optimal step size to update the model through line search.

Images