Data processing and interaction control system based on dynamic eye position tracking

Abstract

The invention discloses a data processing and interaction control system based on dynamic eye position tracking, and particularly relates to the technical field of man-machine interaction and eye movement tracking, which comprises an image acquisition module, a data processing module, an interaction control module and an equipment interface module. The image acquisition module controls a near-infrared camera to capture an eye video; the data processing module executes dynamic calibration, triggers calibration points based on a historical gaze point distribution sparse region, updates a sight line mapping model online by adopting a recursive least square method, and recognizes an interaction intention through a state machine in combination with a gaze point sequence and an eye movement event mode; the interaction control module carries out logic binding on the eye movement intention and signals such as head postures, gestures and voice, and a composite control instruction is generated through complementary fusion, concurrent binding and conflict arbitration; and the device interface module standardizes the instruction and then outputs the instruction to the controlled device. The system realizes non-intrusive dynamic calibration, effectively improves tracking precision and interaction robustness, and is suitable for a multi-modal fusion control scene.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and eye-tracking technology, and more specifically, to a data processing and interactive control system based on dynamic eye tracking. Background Technology

[0002] In existing technologies, eye tracking, as a natural human-computer interaction method, has been widely applied in fields such as assisted communication, virtual reality, user experience research, and intelligent device control. Traditional eye tracking systems typically rely on a pre-executed, static calibration process. During this process, the user needs to sequentially gaze at a series of pre-defined fixed points on the screen, and the system uses this to establish a mapping model between the user's eye features and screen coordinates (i.e., a gaze mapping model). This system captures images of the user's eyes, extracts features such as the pupil center and corneal reflection, and uses a pre-defined model to estimate the user's gaze point, thereby recognizing basic interaction intentions such as gazing and blinking.

[0003] However, in actual use, it still has some drawbacks, such as the cumbersome and intrusive calibration process that interrupts the user's main task flow and results in a poor experience; once the user's head position or posture moves, or the device slides slightly, the pre-established calibration model will fail, causing a sharp drop in tracking accuracy, requiring the user to interrupt the current operation for recalibration; existing eye-tracking interaction systems mostly rely on single gaze point information, have limited ability to recognize user intentions, are prone to accidental triggering due to unconscious gaze wandering, and are difficult to execute complex interactive commands. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, the present invention provides a data processing and interactive control system based on dynamic eye tracking, which solves the problems mentioned in the background art through the following scheme.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a data processing and interactive control system based on dynamic eye tracking, comprising: Image acquisition module: configured to control the image sensor to capture a video image stream of the user's eye; Data processing module: configured to process the video image stream to generate calibrated gaze point data, and specifically used for: Perform dynamic calibration operation: During the main task of system operation, based on the user's gaze behavior to randomly appearing calibration points, the gaze mapping model is updated online through parameter estimation algorithm; wherein, the location of the calibration point is based on real-time analysis of the spatial distribution of the user's historical gaze point trajectory, and priority is given to covering screen areas with sparse historical gaze point distribution. Perform multimodal intent recognition operation: using the updated gaze mapping model, process the raw eye image data obtained from the video image stream and output a calibrated gaze point coordinate sequence; through a state machine, perform spatiotemporal correlation analysis between the calibrated gaze point coordinate sequence and a predefined eye movement event pattern, and combine it with the operable state of the current focus user interface element to jointly infer the user's interaction intent; Interactive control module: configured to logically bind the interactive intent inferred from the multimodal intent recognition operation with signals from other input modalities to generate composite control commands; Device interface module: configured to output the composite control command to drive the controlled device to perform corresponding operations.

[0006] Preferably, the image sensor includes a near-infrared camera used in conjunction with an infrared illumination source; the image acquisition module is further configured to dynamically adjust the parameters of the image sensor, including exposure time, gain, frame rate, and resolution, wherein the adjustment of the exposure time and gain is calculated based on a comparison between the average brightness of the current frame image and a preset target brightness, the frame rate is set between 30 frames / second and 120 frames / second, and the resolution is at least 640×480 pixels.

[0007] Preferably, the screen area with sparse distribution of historical gaze points includes: dividing the screen into an M×N grid and calculating the number of historical gaze points in each grid cell in real time; based on the number of historical gaze points, determining the grid cells with a number of historical gaze points lower than the average value as sparsely distributed areas; the calibration point is randomly selected from the sparse area, and the probability of its selection is inversely proportional to the number of historical gaze points in that grid cell.

[0008] Preferably, the parameter estimation algorithm includes: calculating the prior estimation error, calculating the gain vector, updating the parameter estimate, and updating the error covariance matrix, wherein a forgetting factor with a value between 0.95 and 0.99 is used to assign weights to the newly collected data.

[0009] Preferably, the state machine includes: an idle state, a gaze start state, a continuous gaze state, an intent-triggered state, and a blink recognition state; wherein, the condition for transitioning from the idle state to the gaze start state is: the gaze point remains within the bounding box of a certain user interface element for a duration exceeding a first preset threshold; the condition for transitioning from the gaze start state to the continuous gaze state is: the gaze point remains continuously within the element, and the total dwell time exceeds a second preset threshold.

[0010] Preferably, the predefined eye movement event pattern includes: intentional blinking pattern and specific trajectory pattern; wherein, the intentional blinking pattern is defined as a blink detected during continuous gaze with a duration in the range of 100 milliseconds to 400 milliseconds; the specific trajectory pattern is defined as a motion trajectory formed by a gaze point coordinate sequence within a specific time window, with a dynamic time warping distance between the trajectory and the pre-stored template trajectory being less than a preset threshold.

[0011] Preferably, the logical binding strategy includes: Complementary fusion: The spatial coordinates of multiple input signals are weighted and averaged with their respective confidence scores to generate fused spatial coordinates; Concurrent binding: When the difference between the timestamps of multiple input signals is less than a time synchronization threshold, these signals are bound together to generate a composite instruction; Conflict arbitration mechanism: When multiple input signals have logical conflicts, the confidence level of each signal is adjusted by a conflict adjustment coefficient that is negatively correlated with the degree of conflict.

[0012] Preferably, the composite control command includes: mapping an abstract command type to a specific operation code supported by the target device by querying a preset command mapping table; and converting normalized input parameter values ​​into device-specific physical ranges by using a linear transformation formula.

[0013] Preferably, the device interface module includes: monitoring device status using polling or event-driven methods, and using a heartbeat mechanism for connection detection; automatically retransmitting instructions and resetting connections in case of errors, and using a selective retransmission protocol to ensure transmission; coordinating timing and resources when controlling multiple devices, and selecting devices based on capability matching and load conditions; hardware equipped with interface circuits, and software using a layered design to shield hardware differences.

[0014] The technical effects and advantages of this invention are as follows: 1. By intelligently triggering calibration points in sparse areas of the screen during the system's main task operation, based on the spatial distribution of the user's historical gaze points, and using parameter estimation algorithms such as recursive least squares to update the gaze mapping model online. This method eliminates the need for users to interrupt their tasks for tedious static calibration and can automatically compensate for errors caused by head movements, device sliding, etc., thus maintaining high-precision eye tracking throughout long-term use, resulting in a smoother and more natural user experience. 2. By using a state machine to perform spatiotemporal correlation analysis between the calibrated gaze point coordinate sequence and predefined eye-tracking events, and closely combining this with the operable state of the currently focused UI element, the interaction intent is inferred. This mechanism goes beyond single gaze point judgment, can more accurately distinguish between conscious interaction behavior and unconscious gaze wandering, reduces the false trigger rate, and supports more complex eye-tracking interaction commands; 3. The interaction control module logically binds eye-tracking interaction intentions with signals from other input modalities such as head posture, gestures, and voice. It generates high-confidence composite control commands through strategies such as complementary fusion, concurrent binding, and conflict arbitration. This makes eye tracking no longer an isolated interaction channel, but integrated into a unified interaction context, enabling it to adapt to a wider range of application scenarios and complex control tasks. At the same time, the system allows for flexible customization of interaction logic for different applications through a configurable rule engine, resulting in strong scalability. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the overall structure of the present invention.

[0016] Figure 2 This is a schematic diagram of the data flow structure between modules in this invention.

[0017] Figure 3 This is a schematic diagram of the dynamic calibration and multimodal intent recognition structure of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] refer to Figures 1-3 The data processing and interactive control system based on dynamic eye tracking shown includes: an image acquisition module, a data processing module, an interactive control module, and a device interface module. The connection method is as follows: the image acquisition module and the data processing module are interconnected, the data processing module and the interactive control module are interconnected, and the interactive control module and the device interface module are interconnected.

[0020] The image acquisition module is configured to control the image sensor to capture a video image stream of the user's eye. The image sensor uses a near-infrared camera, in conjunction with an infrared illumination source (such as an infrared LED), to clearly capture eye images under various lighting conditions, including low-light environments, while minimizing interference from ambient light. Near-infrared light is invisible to the human eye, therefore it does not affect the user experience.

[0021] The image acquisition module interacts with the image sensor via software, controlling its startup, shutdown, and parameter adjustments. This software driver can be implemented based on the operating system's camera API. The module dynamically adjusts the image sensor's parameters, including exposure time, gain, frame rate, and resolution, to optimize image quality.

[0022] In automatic exposure control, the module adjusts the exposure time based on the average brightness of the image. The average brightness of the current frame is calculated. and target brightness Compare:

[0023] in, It represents the total number of pixels in an image, indicating the total number of pixels in a single frame. It is the first The brightness value of each pixel (for grayscale images, or values ​​converted to grayscale) typically ranges from 0 to 255 (for 8-bit images). For pixel index. Target brightness. Setting it to 128 (for 8-bit images) represents medium brightness, ensuring the image is neither too bright nor too dark. New exposure time. The calculation is as follows:

[0024] in, This is the current exposure time; This is the adjusted exposure time. To avoid over-adjustment, Limited to the minimum exposure time allowed by the sensor and maximum exposure time These values ​​are determined by the sensor hardware specifications.

[0025] Gain adjustment is based on brightness error. New gain. The calculation is as follows:

[0026] in, This is the current gain value; This is the adjusted gain value. It is also limited to the sensor's minimum gain. and maximum gain Within a certain range to prevent signal distortion.

[0027] The frame rate is set between 30 frames per second and 120 frames per second to ensure that rapid eye movements can be captured. The actual frame rate may be dynamically adjusted based on the system's processing power, but is typically fixed at 60 frames per second to balance accuracy and performance. The parameter represents the number of frames captured per second and is jointly controlled by the sensor hardware and software drivers.

[0028] The resolution should be at least 640×480 pixels, but higher resolutions can be used to improve tracking accuracy. 1280×720 pixels is preferred for improved tracking accuracy.

[0029] The image acquisition module also monitors image quality in real time and triggers recalibration or parameter adjustments when image quality deteriorates (e.g., due to motion blur or defocus). Furthermore, the module can include preliminary image preprocessing functions, such as noise suppression, contrast enhancement, or image cropping, to improve image quality and reduce data transmission. These preprocessing functions can be implemented using hardware acceleration (e.g., GPUs) or software algorithms (e.g., the OpenCV library).

[0030] In noise suppression, Gaussian filtering may be applied. The kernel size of the Gaussian filter is set to 3×3 or 5×5, and the standard deviation is... Set to 1.0 to control the filtering strength. The filtered image. The calculation is as follows:

[0031] in, It is the Gaussian kernel radius, for a 3x3 kernel. For 5x5 cores, ; The input image is at position Pixel values; It is the Gaussian kernel function, defined as:

[0032] in, It is the standard deviation, which affects the smoothness of the filter.

[0033] In terms of hardware, image sensors can be integrated into user equipment or connected as external devices via standard interfaces. The image acquisition module ensures that video image streams are continuously transmitted to the data processing module with low latency, providing a reliable data source for subsequent processing.

[0034] It should be further explained that the image acquisition module uses near-infrared illumination and a camera primarily because near-infrared light is invisible to the human eye and will not cause user discomfort. It also provides clear image contrast in low-light environments, facilitating subsequent eye feature extraction. The target brightness is set to 128 (for 8-bit images) based on empirical values ​​to ensure moderate overall image brightness and avoid overexposure or underexposure. Automatic exposure and gain adjustment employ a proportional control algorithm for rapid response to changes in lighting conditions. Gaussian filtering is used for noise suppression; the kernel size and standard deviation are chosen to balance noise smoothing and detail preservation. A kernel size of 3×3 or 5×5 represents a trade-off between computational efficiency and filtering effectiveness. The frame rate is set to 60 frames per second to capture rapid eye movements without excessively consuming computational resources. The resolution of 1280×720 pixels is chosen to balance accuracy and performance, considering both processing speed and feature recognition accuracy.

[0035] The data processing module is configured to process the video image stream to generate calibrated gaze point data, and is specifically used for: Perform dynamic calibration: Dynamic calibration is performed continuously during the main system task. Its purpose is to update the gaze mapping model online to compensate for errors caused by user head movement, device slippage, or environmental changes.

[0036] The strategy for generating calibration points is crucial. The system maintains a set of historical gaze point coordinates. ,in and These are the horizontal and vertical coordinates of the user's gaze point estimated in the screen coordinate system over a past period (e.g., the last 30 seconds). The system calculates these coordinates in real time. Spatial density distribution. The screen is divided into a grid, with each grid cell... Number of historical gaze points The calculation is as follows:

[0037] in, As an indicator function, when the gaze point Falling into the grid cell The value is 1 if the condition is met, and 0 otherwise. Parameter This indicates the historical gaze frequency of the grid cell; a low value indicates that there are few users looking at the area and a high need for calibration.

[0038] Location of the next calibration point Random selection is made from sparsely distributed grid cells representing historical gaze points. Specifically, the selection probability... and Proportional, that is:

[0039] This strategy ensures that calibration points prioritize covering underexplored areas of the screen, thus enabling efficient and non-invasive calibration.

[0040] When a user gazes at a calibration point, the system collects a set of corresponding eye feature data. (e.g., pupil center coordinates, corneal reflector coordinates, etc.) and known screen calibration point coordinates The gaze mapping model typically employs a multinomial regression model, such as a second-order multinomial:

[0041] in, These are the screen coordinates predicted by the model; These are the normalized coordinates of the eye features; and These are model parameters.

[0042] The system uses recursive least squares to update these parameters online. Taking the horizontal coordinate model as an example, the parameter vector is defined. and eigenvectors The update steps of the RLS algorithm are as follows: Calculate the prior estimation error:

[0043] Calculate the gain vector:

[0044] Update parameter estimates:

[0045] Update the covariance matrix:

[0046] in, It is the error covariance matrix, initially a large diagonal matrix; It is the forgetting factor, with a value range of (0,1] (usually set to ). This is used to assign higher weights to new data, thereby enabling the model to adapt to changes.

[0047] Perform multimodal intent recognition: This operation utilizes a dynamically calibrated gaze mapping model to convert raw eye image data into a calibrated gaze point coordinate sequence. ,in It is a moment The coordinates of the gaze point.

[0048] Intent recognition is achieved through a state machine that stores the gaze sequence. Spatiotemporal correlation analysis is performed with predefined eye-tracking event patterns, and the interaction intent is inferred by combining the operable state of the currently focused user interface element.

[0049] The state machine includes at least the following states: idle state, gaze initiation state, continuous gaze state, intention-triggered state, and blink recognition state. The state transition conditions are as follows: The condition for transitioning from the idle state to the gaze initiation state is: the gaze point. In a certain UI element Duration of stay within the bounding box Exceeding the preset threshold (e.g., 200 milliseconds). Parameters This indicates the cumulative duration of continuous staring.

[0050] The condition for transitioning from the initial gaze state to the sustained gaze state is that the gaze point remains continuously on the element. Inside, and total stay time (e.g., 500 milliseconds).

[0051] The condition for transitioning from a sustained gaze state to an intention-triggered state is that the system detects a specific eye-movement event that matches the "confirmation" intention. This event can be: Intentional blinking: During sustained gaze, one blink of sustained duration was detected. exist blinking within range ( ). It is detected by analyzing mutations in eye features such as pupil aspect ratio.

[0052] Specific trajectory: gaze point sequence In the time window A trajectory matching the pre-stored pattern is formed. The matching degree is calculated by determining the dynamic time-warped distance between the two trajectories. To measure, when Less than the threshold When the match is successful, it is considered a successful match.

[0053] At each step of the state transition, the system incorporates the currently focused UI element. The system comprehensively assesses the operable states. Finally, when the state machine enters the intent-triggered state, it outputs an interaction intent corresponding to the UI element and the eye-tracking event pattern. This intent will be passed to the interaction control module for further processing.

[0054] It should be further explained that in the dynamic calibration operation, the calibration point generation strategy is based on the historical gaze point distribution, prioritizing coverage of sparse areas. This is because the gaze mapping model may not be accurate enough in areas where the user gazes less. By triggering calibration points in these areas, the model accuracy can be improved efficiently. Recursive least squares was chosen because it can recursively update model parameters without storing all historical data, making it suitable for real-time systems. The forgetting factor was set to 0.95 to 0.99 to emphasize the impact of new data, enabling the model to adapt quickly to changes. In multimodal intent recognition, the state machine design simulates the human attention mechanism, identifying intentional interactions through gaze duration and specific eye movement events. Thresholds such as 200 milliseconds and 500 milliseconds are based on user experience research, ensuring the system is neither overly sensitive nor unresponsive. The blink duration threshold (100-400 milliseconds) is based on physiological characteristics, excluding unintentional blinks. Dynamic time warping is used for trajectory matching because it can handle the scaling of time series, improving the robustness of pattern recognition.

[0055] The interactive control module is configured to logically bind the interactive intent inferred from the multimodal intent recognition operation with signals from other input modalities to generate composite control commands. This module continuously receives various input signals from upstream modules and performs preprocessing. The main input signals include eye-tracking interaction intentions from the data processing module, which are represented as tuples. .in, Intent type; For the purpose of action; Confidence of eye-tracking intention The value ranges from [0,1] and is calculated by the data processing module based on the state machine matching degree and the quality of the eye signal.

[0056] Simultaneously, the module also receives various input modal signals from other system interfaces, such as head pose data. Gesture data Voice commands And traditional input device signals such as .

[0057] All input signals are fed into a unified event bus and are timestamped and normalized to ensure they are processed within a unified time frame and data structure.

[0058] This module includes a built-in configurable rule engine for defining the logical binding relationships between different input signals. Based on the current application context, this engine performs signal fusion and generates composite instructions. Its fusion strategies mainly include the following: Complementary fusion: When multiple input signals have complementary characteristics in the spatial or temporal dimensions, a weighted fusion algorithm is used to improve overall accuracy. The spatial coordinates of the fused output are calculated by weighting the spatial coordinates of each input signal with its confidence score.

[0059] in: Represents the merged spatial coordinates. Indicates the first The spatial coordinates of the input signal Indicates the first Confidence level of each input signal This represents the corresponding weighting coefficient.

[0060] Concurrent binding: When signals of multiple modes are within a time window When events occur almost simultaneously, a logical binding relationship is established through time synchronization detection, allowing the generation of a more complex instruction. The basic conditions for binding can be expressed as follows:

[0061] in, and These represent the timestamps of different input signals. Indicates the time synchronization threshold. and These represent the minimum confidence thresholds for each signal.

[0062] Conflict arbitration mechanism: When multiple input signals conflict logically, an arbitration strategy based on confidence level and priority is adopted. The adjusted confidence level can be calculated by multiplying the original confidence level by the conflict factor.

[0063] in, This indicates the adjusted confidence level. Indicates the original confidence level. This represents the conflict adjustment coefficient, whose value is negatively correlated with the degree of conflict between signals.

[0064] Composite instruction generation: Based on the fusion processing results, the module generates structured composite control instructions. The overall confidence level of the composite instruction is calculated by weighting the confidence levels of each participating signal.

[0065] in Indicates the overall confidence level of the compound instruction. Indicates the first The weighting coefficients of each signal. Indicates the first The confidence level of a signal.

[0066] The data structure of a compound instruction includes an instruction type field, a target identifier field, a confidence level field, and a parameter set field. The parameter set is used to store various auxiliary parameters required to execute the instruction.

[0067] At the hardware level, this module can run on the system's central processing unit. At the software level, it can be implemented as an independent system service, providing various applications with the ability to register interaction rules and receive control commands through an application programming interface (API). The rule engine can be implemented using lightweight scripting languages ​​or declarative configuration files, allowing for flexible customization of interaction logic for different applications.

[0068] It should be further explained that the interaction control module employs a multimodal signal fusion strategy to improve the accuracy and robustness of the interaction. Complementary fusion combines the spatial coordinates of each modality through a weighted average, with weights based on confidence levels, thus providing stable output even when a single modality is unreliable. The time synchronization threshold for concurrent binding is typically set to 100-200 milliseconds, based on research into human multimodal perception, which indicates that events within this time window are considered to occur simultaneously. The conflict arbitration mechanism handles conflicts by adjusting confidence levels; the conflict adjustment coefficient is negatively correlated with the degree of conflict between signals, ensuring that high-confidence signals dominate decision-making. The rule engine uses scripts or configuration files, enabling the system to flexibly adapt to the interaction logic of different applications; for example, different binding rules can be defined in a virtual reality environment. The confidence calculation of composite instructions is a weighted sum, with weight coefficients adjusted according to the application scenario to reflect the importance of each modality.

[0069] Device interface module: configured to output the composite control command to drive the controlled device to perform corresponding operations.

[0070] After receiving composite control commands from the interactive control module, the module first performs command standardization processing. This processing includes command format conversion, parameter range normalization, and timing adjustment.

[0071] For instruction format conversion, the abstract instruction type in the composite control instruction is mapped to the specific opcode supported by the target device. This mapping is achieved by querying a preset instruction mapping table:

[0072] in Indicates the target device's operation code. Indicates the type of abstract instruction input. Indicates the type of target device. This represents the instruction mapping function.

[0073] For parameters with regular ranges, the normalized parameter values ​​are converted into equipment-specific physical ranges. A linear transformation formula is used:

[0074] in Indicates the physical quantity value of the equipment. and These represent the minimum and maximum ranges of the device parameters, respectively. This represents the normalized input parameter value.

[0075] The standardized instructions are encapsulated according to the communication protocols supported by the target device. The encapsulation process includes data frame assembly, checksum calculation, and transmission timing control.

[0076] The data frame structure includes a frame header, device address, opcode, parameter fields, and checksum. The checksum is calculated using a cyclic redundancy check algorithm.

[0077] in Indicates the first in the data frame The value of 1 byte, Indicates the total length of the data frame. Indicates the cardinality of the generator polynomial. Represents the modulus.

[0078] For control commands with high real-time requirements, a priority scheduling algorithm is used to ensure timely transmission:

[0079] in Indicates scheduling priority. Indicates the basic priority. Indicates the weight of urgency. Indicates the urgency of the sequence. Represents time-series weights.

[0080] The module continuously monitors the status of the controlled devices, including connection status, response status, and error status. Device status is obtained through periodic polling or event-driven methods.

[0081] Connection status detection uses a heartbeat mechanism, with the heartbeat interval dynamically adjusted based on the device type.

[0082] in Indicates the interval between heartbeats. Indicates the basic interval. Represents the network latency coefficient. This indicates the packet loss rate.

[0083] Error status detection is achieved by analyzing error codes and abnormal signals returned by the device. When an error status is detected, an error recovery process is automatically initiated, including command retransmission and connection reset.

[0084] To ensure the reliability of command transmission, the module implements a multi-layered error detection and correction mechanism, including a data packet acknowledgment mechanism at the transport layer and a command execution acknowledgment mechanism at the application layer.

[0085] The data packet acknowledgment mechanism employs a selective retransmission protocol, which retransmits only the abnormal data packets when packet loss or corruption is detected.

[0086] in Indicates the maximum number of retransmissions. Indicates the maximum number of retransmissions allowed by the system. Indicates the target transmission success rate. This indicates the network packet loss rate.

[0087] When the system needs to control multiple devices simultaneously, the module is responsible for coordinating the timing of instructions and resource allocation among the devices. The device resource management table records the occupancy status and capability characteristics of each device to avoid resource conflicts and instruction interference.

[0088] The equipment selection strategy is based on a comprehensive evaluation of equipment capacity matching and current load conditions:

[0089] in This indicates the equipment selection score. Indicates the ability matching weight. Indicates the degree of ability matching. Indicates load weight. This indicates the current load rate.

[0090] At the hardware level, this module configures the corresponding physical interface circuits based on the target device's interface type, including level conversion, signal isolation, and driver amplification circuits. At the software level, a layered design of device abstraction layer and driver layer is adopted, shielding the underlying hardware differences through a unified device management interface.

[0091] It should be further explained that the device interface module, through instruction standardization, converts abstract instructions into device-specific opcodes. This is achieved through a pre-defined instruction mapping table, enabling the system to support multiple devices without modifying the core code. Parameter normalization uses linear transformation, which is simple and efficient, mapping normalized parameters to the device's physical range. Data encapsulation uses Cyclic Redundancy Check (CRC) to detect transmission errors and ensure data integrity. The priority scheduling algorithm considers the urgency and timing of instructions, ensuring timely transmission of critical instructions. The heartbeat mechanism dynamically adjusts the interval based on network latency and packet loss rate to balance connection detection overhead and response speed. Error recovery mechanisms, such as selective retransmission, improve system reliability. The device selection strategy is based on capacity matching and load conditions, ensuring reasonable resource allocation and avoiding overload.

[0092] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data processing and interactive control system based on dynamic eye position tracking, characterized in that, Comprise: An image acquisition module configured to control an image sensor to capture a video image stream of a user's eye; A data processing module configured to process the video image stream to generate calibrated gaze point data, and specifically for: Performing a dynamic calibration operation: during the system running a main task, based on the user's gaze behavior on randomly appearing calibration points, updating the line-of-sight mapping model online through a parameter estimation algorithm; wherein the appearance position of the calibration points is based on real-time analysis of the spatial distribution of the user's historical gaze point trajectory, and preferentially covers the screen area where the historical gaze point distribution is sparse; Performing a multi-modal intent recognition operation: using the updated line-of-sight mapping model, processing the raw eye image data obtained from the video image stream to output a calibrated gaze point coordinate sequence; through a state machine, spatio-temporal correlation analysis of the calibrated gaze point coordinate sequence and the pre-defined eye movement event pattern, and combining the operable state of the current focus user interface element, jointly inferring the user's interaction intent; An interaction control module configured to logically bind the interaction intent inferred by the multi-modal intent recognition operation with signals from other input modalities to generate a composite control instruction; A device interface module configured to output the composite control instruction to drive the controlled device to perform corresponding operations.

2. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The image sensor comprises a near-infrared camera used in conjunction with an infrared illumination source; the image acquisition module is further configured to dynamically adjust the parameters of the image sensor, including exposure time, gain, frame rate and resolution, wherein the exposure time and gain are adjusted based on the comparison of the average brightness of the current frame image with the pre-set target brightness, the frame rate is set between 30 frames / second and 120 frames / second, and the resolution is at least 640x480 pixels.

3. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The screen area where the historical gaze point distribution is sparse comprises dividing the screen into an MxN grid and calculating the number of historical gaze points in each grid cell in real time; based on the number of historical gaze points, the grid cells with a number of historical gaze points lower than the average are determined as the sparse area; the calibration points are randomly selected from the sparse area, and the probability of being selected is inversely proportional to the number of historical gaze points in the grid cell.

4. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The parameter estimation algorithm comprises calculating the prior estimation error, calculating the gain vector, updating the parameter estimation, and updating the error covariance matrix, wherein a forgetting factor with a value range of 0.95 to 0.99 is used to assign weights to newly collected data.

5. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The state machine comprises an idle state, a gaze start state, a sustained gaze state, an intent trigger state, and a blink recognition state; wherein the condition for transferring from the idle state to the gaze start state is that the gaze point stays within the bounding box of a certain user interface element for more than a first pre-set threshold; the condition for transferring from the gaze start state to the sustained gaze state is that the gaze point stays within the element and the total stay time exceeds a second pre-set threshold.

6. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The predefined eye movement event patterns include an intentional blink pattern and a specific trajectory pattern; the intentional blink pattern is defined as a blink with a duration in the range of 100-400 ms detected during a sustained gaze; the specific trajectory pattern is defined as a motion trajectory formed by a sequence of gaze point coordinates within a specific time window, with a dynamic time warping distance to a pre-stored template trajectory less than a preset threshold.

7. The dynamic eye position tracking based data processing and interaction control system of claim 1, wherein, The strategy of logical binding includes: Complementary fusion: the spatial coordinates of multiple input signals are weighted and averaged with their respective confidence levels to generate fused spatial coordinates; Concurrent binding: when the difference between the timestamps of multiple input signals is less than a time synchronization threshold, these signals are bound to generate a composite instruction; Conflict arbitration mechanism: when there is a logical conflict among multiple input signals, the confidence level of each signal is adjusted by a conflict adjustment coefficient negatively related to the degree of conflict.

8. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The composite control instruction includes: mapping the abstract instruction type to the specific operation code supported by the target device through querying a preset instruction mapping table; converting the normalized input parameter value to the device-specific physical range through a linear transformation formula.

9. The data processing and interaction control system based on dynamic eye position tracking of claim 1, wherein, The device interface module includes: a heartbeat mechanism for connection detection through polling or event-driven monitoring of device status; a selective retransmission protocol for ensuring transmission through automatic retransmission of instructions and resetting of the connection in case of error; coordination of timing and resources when controlling multiple devices, with the devices selected based on the ability matching degree and load status score; a hardware interface circuit, and a layered software design to shield hardware differences.

Images