Multi-eye tracker data processing method and device based on common port communication

Abstract

The invention belongs to the technical field of eye tracking, particularly relates to a multi-eye tracker data processing method and device based on common port communication, and aims to solve the problems of high communication overhead, difficulty in time synchronization and low data fusion precision during cooperative work of multiple eye trackers in the prior art. The method provided by the invention comprises the following steps: respectively constructing logic channels through a common communication port according to received communication data of a plurality of eye trackers; performing time synchronization on the plurality of eye trackers based on the global time reference, and scheduling the eye trackers to orderly feed back calibration data; and determining global calibration parameters based on the calibration data to establish a global calibration model, and performing coordinate correction on subsequently acquired eye movement data based on the model. According to the method, through single-port multi-channel multiplexing, global synchronous scheduling and fusion modeling, the system resource consumption is remarkably reduced, millisecond-level equipment collaboration is realized, and the overall analysis precision of eye movement data is greatly improved.

Description

Technical Field

[0001] This application belongs to the field of eye-tracking technology, specifically relating to a multi-eye tracker data processing method and apparatus based on shared port communication. Background Technology

[0002] Eye-tracking technology captures and analyzes a user's gaze location information in real time, accurately reconstructing the user's gaze behavior. It has been widely used in various fields such as aerospace, human-computer interaction, psychological research, and market analysis. In complex scenarios such as cockpit operation simulation and large-size display panel interaction, a single eye tracker is limited by its own field of view and measurement accuracy, and cannot achieve high-precision gaze tracking with full coverage. Therefore, the collaborative operation of multiple eye trackers has become an important development trend in the field of eye-tracking.

[0003] However, existing multi-eye tracker collaborative technologies generally suffer from the following problems: First, in terms of communication architecture, each eye tracker is typically assigned an independent communication port and control thread. As the number of devices increases, this approach rapidly consumes server resources such as ports and threads, and generates significant redundant network communication overhead, especially when distributing common parameters such as calibration point coordinates. Second, regarding the collaborative working mechanism, the lack of a unified global time reference and efficient scheduling strategy leads to significant time discrepancies between eye trackers when initiating calibration or data acquisition. This makes it difficult to accurately align and fuse the acquired data in the time dimension, directly affecting the accuracy of subsequent data analysis. Finally, in terms of data processing, traditional methods often simply stitch together or selectively use data from different devices. For data quality issues in overlapping areas of the device's field of view, especially at the screen edges, there is a lack of effective fusion and correction mechanisms, resulting in low accuracy of the final generated eye-tracking trajectory.

[0004] Therefore, how to design an efficient and low-overhead communication mechanism, and on this basis achieve accurate synchronization, collaborative calibration and high-quality data fusion of multiple eye trackers, is a technical problem that urgently needs to be solved in the field of eye tracking technology. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, namely the high communication overhead, difficulty in time synchronization, and low data fusion accuracy when multiple eye trackers work together, the first aspect of this application proposes a multi-eye tracker data processing method based on shared port communication, the method comprising the following steps:

[0006] Using a shared communication port, logical channels are constructed for each eye tracker based on the communication data received from multiple eye trackers. Multiple eye trackers are synchronized based on a global time reference, and based on the time synchronization results, each eye tracker is scheduled to feed back calibration data in an orderly manner through its logical channel within different time windows. Based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data, global calibration parameters are determined to establish a global calibration model; Based on the global calibration model, coordinate correction is performed on the subsequently acquired real-time eye-tracking data to generate eye-tracking coordinates.

[0007] In some preferred embodiments, determining the global calibration parameters includes: Based on the generation timestamp of the calibration data, the weight corresponding to the timeliness of the data is calculated using an exponential decay function; The weights corresponding to spatial calibration errors are determined based on the standard deviation between the coordinates measured by the eye tracker and the actual coordinates of the target calibration point.

[0008] In some preferred embodiments, the acquired eye-tracking data is corrected, specifically as follows: For eye-tracking data points located in the preset screen edge area, based on the spatial distance between the eye-tracking data point and the center anchor point of the effective area of ​​each eye tracker, the calibration completion degree of each eye tracker, and the density coefficient of the adjacent calibration points, corresponding optimization weights are generated for each eye tracker. The coordinates of the eye-tracking data points reported by each eye tracker are weighted and averaged using the optimized weights to obtain the corrected coordinates.

[0009] In some preferred embodiments, the construction of the respective logical channels specifically includes: Establish communication connections between the server and multiple eye trackers, parse the device identifiers contained in the received communication data, and divide the shared communication port into multiple logical channels according to the device identifiers, with each device identifier corresponding to a logical channel.

[0010] In some preferred embodiments, priority information is embedded in the device identification field, and the method further includes: Preemptive transmission scheduling is performed on different types of data based on the priority information to ensure low-latency transmission of critical instructions.

[0011] In some preferred embodiments, the method further includes: The system monitors the data transmission delay of each logical channel in real time, and adaptively increases the priority of the time reference packet transmitted on that logical channel when the delay of any logical channel exceeds a preset threshold.

[0012] In some preferred embodiments, before scheduling the calibration data from each eye tracker, the method further includes: A time-slice round-robin scheduling strategy is adopted, and different start-up times are allocated to each eye tracker according to the device identifier to achieve orderly start-up of collaborative calibration.

[0013] In some preferred embodiments, before scheduling the calibration data from each eye tracker, the method further includes: Based on the historical calibration error of each eye tracker, the dwell time at the calibration point is adaptively adjusted. Common calibration parameters, including calibration point coordinates, are broadcast to all eye trackers at once.

[0014] A second aspect of this application proposes a multi-eye tracker data processing device based on shared port communication, comprising: The communication management unit is configured to construct separate logical channels for multiple eye trackers based on the communication data received from multiple eye trackers via a shared communication port. The collaborative scheduling unit is configured to synchronize the time of multiple eye trackers based on a global time reference, and according to the time synchronization results, schedule each eye tracker to feed back calibration data in an orderly manner through its logical channel within different time windows. The model generation unit is configured to determine calibration parameters based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data, so as to establish a global calibration model; The coordinate correction unit is configured to perform coordinate correction on the subsequently acquired real-time eye-tracking data based on the global calibration model to generate eye-tracking coordinates.

[0015] Compared with the prior art, the embodiments of this application have at least the following beneficial effects: 1) By adopting the design of "single physical port + multiple logical channels" and combining common parameter broadcasting and incremental data transmission strategies, the server port occupation and network communication redundancy are significantly reduced, the overall system resource consumption is reduced, and the communication efficiency is improved. By establishing a global calibration model that comprehensively considers spatial error and timeliness, the coordinate accuracy of eye-tracking data, especially edge region data, is effectively improved, providing a high-quality data source for subsequent accurate analysis. 2) Based on the global time base and time slice round-robin scheduling mechanism, the start-up deviation of multiple devices is controlled within milliseconds, effectively avoiding data congestion, and targeted compensation is provided for devices with poor calibration performance, which provides a solid foundation for the accurate fusion of subsequent data in the time dimension, greatly improving the success rate and reliability of collaborative calibration and increasing the calibration success rate. 3) For edge region data, a multi-dimensional weighted optimization algorithm is introduced, which integrates three dimensions: "spatial distance, calibration completion, and calibration point density" to effectively reduce the overall eye-tracking coordinate error and improve the efficiency of edge data. By combining the broadcasting of common parameters, the incremental transmission of dynamic data, and the adaptive flow control strategy, network communication redundancy is greatly reduced. Attached Figure Description

[0016] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of a multi-eye tracker data processing method based on shared port communication provided in one embodiment of this application; Figure 2 This is a schematic diagram of the application architecture of a multi-eye tracker data processing device based on shared port communication according to an embodiment of this application; Figure 3 This is an eye-tracking trajectory diagram after device calibration following the application of a multi-eye tracker data processing method based on shared port communication, provided in one embodiment of this application. Figure 4 This is a schematic diagram of the structure of a computer system used to implement the methods, apparatus, and electronic devices of this application. Detailed Implementation

[0017] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0018] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] To address the technical problems of existing technologies, such as high communication overhead, difficulties in time synchronization, inaccurate data fusion, and poor data quality at screen edges, this application provides a multi-eye tracker data processing method based on shared port communication. This method constructs isolated logical channels for multiple eye trackers via a shared communication port, effectively reducing communication resource overhead. Based on a unified global time reference, it performs high-precision time synchronization and collaborative scheduling for each eye tracker, solving the timing challenges of multi-device collaborative work. Furthermore, it establishes a more accurate global calibration model by combining spatial calibration errors and data timeliness. Based on this model, it optimizes and corrects eye-tracking data in the screen edge region by introducing multi-dimensional weights, significantly improving the overall data quality.

[0020] To more clearly explain the multi-eye tracker data processing method based on shared port communication in this application, the following will be combined with... Figure 1 and Figure 2 The steps in the embodiments of this application are described in detail.

[0021] Please see Figure 2 This document illustrates the application architecture of a multi-eye tracker data processing device based on a shared communication port, as provided in an embodiment of this application. It constructs a four-layer architecture of "client-communication-server-hardware," aiming to optimize the entire process of collaborative control, calibration synchronization, data transmission, and analysis of multiple eye trackers. The descriptions of each layer of the application system architecture are as follows: Client Layer: As the entry point for user interaction, it is responsible for sending control commands to the server layer and receiving and visualizing processing results. In this embodiment, the client layer provides a unified user interface (UI), through which users can select multiple eye trackers that need to work together and start or stop the entire application system. During the calibration process, the client interface displays a calibration progress bar in real time and has a built-in automatic retry mechanism to handle single calibration failures. The position of the calibration point on the interface canvas is determined according to... Deploy proportionally to ensure adaptation to screens of different resolutions. Here, L represents the coordinates of the left edge of the drawn dot grid, T represents the coordinates of the top edge of the drawn dot grid, x represents the relative X coordinate of the calibration point, y represents the relative Y coordinate of the calibration point, W represents the actual width of the calibration canvas, H represents the actual height of the calibration canvas, and 12 is used for centering compensation (a constant that can be adjusted as needed), ensuring that the center point of the dot grid coincides with the calibration point. During calibration, the application system guides users with voice prompts and displays the calibration score upon completion. In the data acquisition phase, the client employs a differentiated labeling strategy, dynamically rendering real-time eye-tracking coordinates with green markers and presenting eye-tracking points in a "staying" state with red markers, enabling intuitive differentiation of eye-tracking states. Furthermore, the client integrates data visualization capabilities, allowing it to access stored eye-tracking data, generate eye-tracking trajectory maps using trajectory clustering algorithms such as K-Means, and allow users to customize time windows to generate heatmaps for specific time periods, achieving integrated "acquisition-analysis-visualization" operations.

[0022] Communication layer: As a bridge for data transmission, it adopts a "single physical port + multiple logical channels" design and is responsible for data frame encapsulation, transmission, verification and flow control between the client, server and hardware.

[0023] Hardware layer: includes multiple eye tracker physical terminals, which serve as the source of data acquisition. Each device is pre-configured with a unique device ID. Server layer: This is the core control layer, which specifically implements the data processing device of this application. It is responsible for listening to the shared port, parsing data, performing time synchronization and collaborative scheduling of multiple devices, establishing a global calibration model, fusing and correcting eye-tracking data, and interacting with the client layer through the communication layer, and ultimately controlling the eye tracker in the hardware layer.

[0024] It should be understood that the core invention of this application lies in the server layer and the data processing method / device executed thereon. In a complete commercial application, this server / device is typically combined with multiple eye trackers (e.g., commercially available eye trackers each configured with a unique device ID) at the hardware layer to form a complete multi-eye tracker data processing application system, in order to achieve collaborative collection and analysis of user eye movement data.

[0025] The multi-eye tracker data processing method based on shared port communication in the first embodiment of this application is applied to a processing device (server). Steps S100-S400 are as follows: Figure 1 As shown, each step is described in detail below: S100: Through a shared communication port, construct separate logical channels for each of the multiple eye trackers based on the communication data received from the multiple eye trackers.

[0026] As an optional implementation, the step of constructing the respective logical channels specifically includes: The system receives communication data from multiple eye trackers, parses the device identifiers contained in the communication data, and divides the shared communication port interface into multiple logical channels based on the device identifiers, with each device identifier corresponding to a logical channel.

[0027] In this embodiment, the server layer occupies only one shared network communication interface (e.g., a UDP port), realizing unified sending and receiving of calibration commands, data and status of multiple devices. Compared with the traditional scheme of allocating independent ports for each device, this greatly reduces the port resource occupancy rate.

[0028] Specifically, the communication layer adopts a "single physical port + multiple logical channels" design. When data frames from different eye trackers arrive at the server through this shared port, the communication processing submodule within the server parses the device ID field contained in the data frame (for example, a 1-byte field that can represent devices from 0x01 to 0x0n). Based on the parsed device ID, the server distributes the data to the corresponding internal processing thread or queue, thereby virtualizing multiple logical channels corresponding one-to-one with the devices on a single physical link, achieving "single-port multi-device data isolation," and the channel switching latency can be controlled within 1ms.

[0029] To improve concurrent processing capabilities, servers can preferably adopt IOCP (I / O Completion Port) technology, which can efficiently manage the network connections of multiple devices through a single thread, avoiding the resource waste of "one thread per device".

[0030] More preferably, to ensure the transmission of critical instructions, the device identification field contains priority information, and the method further includes: performing preemptive transmission scheduling on different types of data according to the priority information to ensure low-latency transmission of critical instructions.

[0031] Specifically, in a preferred embodiment, the high-order portion of the device identification field may selectively embed 2 priority flag bits to define a three-level data priority: High priority (e.g., marked "01") corresponds to critical signaling such as calibration start / stop commands and error over-threshold feedback; medium priority (e.g., marked "00") corresponds to routine data such as real-time eye-tracking coordinates and calibration progress; low priority (e.g., marked "10") corresponds to background data such as device heartbeat packets and historical error statistics. When distributing data, the communication layer follows the principle of "high-priority preemption" to ensure that critical instructions can occupy the channel first and avoid being blocked by ordinary data.

[0032] Further preferably, to ensure the reliability of data transmission and the ease of parsing, this embodiment also customizes the communication frame structure, including: Frame header (2 bytes, e.g., 0xAA55), device ID (1 byte, including priority identifier), data type (1 byte), calibration stage identifier (1 byte, new field, e.g., 0x01=pre-calibration, 0x02=formal calibration, 0x03=data optimization, 0x04=calibration complete), data length (2 bytes), data body (n bytes, can be JSON, XML, or custom binary format), checksum (2 bytes), frame trailer (2 bytes, e.g., 0x55AA).

[0033] Specifically, the checksum is generated using an XOR algorithm, such as: ; in, For checksum, The total number of bytes in the data body. This is the k-th byte of the data body. After receiving the data frame, the client or server performs the same checksum calculation. If the result does not match, a retransmission is requested to ensure data integrity.

[0034] S200 synchronizes multiple eye trackers based on a global time reference, and according to the time synchronization results, schedules each eye tracker to feed back calibration data in an orderly manner through its logical channel within different time windows.

[0035] Preferably, the core control submodule of the server has a built-in unified NTP (Network Time Protocol) time service as a global time reference, which periodically broadcasts time reference packets to all eye trackers; the time reference packets contain the server's current time. And the expected transmission delay threshold. Device Upon receiving a time base packet, record the local reception time. and local time A comparison is made; when a confirmation of receipt is sent to the server, the server records the time of receipt. and the device sends confirmation at its local time. Comparison. The server can calculate uplink and downlink transmission delays by recording the timestamps of device requests and acknowledgments. and Finally, the precise time offset was calculated. and distribute to the equipment i To correct its local clock: ; in, = = ， This is the current local clock time.

[0036] More preferably, in order to address the synchronization error that may be caused by fluctuations in single-port communication bandwidth, this method also monitors the transmission delay of each logical channel in real time. If the delay exceeds the threshold, the server will temporarily increase the transmission priority of the NTP packet (for example, by setting its priority identifier to "01") through feedback via a high-priority channel, so as to ensure that the synchronization error between multiple devices is stable within 1ms.

[0037] With precise time synchronization in place, the calibration task scheduler begins operation. To reduce communication redundancy, common calibration parameters (global common data, such as global calibration coordinates), including calibration point coordinates, are encapsulated into a broadcast frame (e.g., device ID=0xFF) and transmitted to all devices at once, avoiding repeated transmission to each device.

[0038] As an optional implementation, the scheduler employs a time-slice round-robin scheduling strategy, sending a sequence containing the global departure time. The calibration start broadcast frame is generated based on the device ID for each eye tracker. Calculate different startup times to achieve staggered startup and avoid data conflicts. For example, the server issues a global trigger time. The calibration start broadcast frame, device Calculate its own off-peak start time : ; The 50ms interval is the set time slice interval.

[0039] More preferably, the dwell time of each eye tracker at the calibration point is adaptively adjusted based on the historical calibration error of each tracker. For example, the standard deviation of the historical calibration error... Larger equipment, its dwell time It will be extended accordingly: ; in, =500ms is the base time. For equipment standard deviation of error The standard deviation of the error for all devices is the average error; the larger the error, the longer the dwell time.

[0040] S300. Based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data, determine the global calibration parameters to establish a global calibration model.

[0041] As an optional implementation, the step of determining the global calibration parameters includes: determining the global calibration parameters based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data; and the core control submodule of the server fusing the collected multiple sets of calibration coefficients.

[0042] Preferably, a two-dimensional fusion logic of "spatial error + temporal timeliness" is constructed, and a global calibration model is established by fusing calibration coefficients through a weighted average algorithm: ; ; in, The calibration coefficients are the result of a weighted average across multiple devices. For equipment calibration coefficients, For equipment The fusion weight (with a value of 0-1). For equipment Spatial calibration error ( The smaller the value, the higher the spatial precision and the greater the weight. For equipment Spatial calibration error, For equipment The weight of the timestamp consistency (data timeliness) of the calibration coefficient (values ​​range from 0 to 1). For equipment Calibration coefficient time difference ( The current time of the server. For equipment (Coefficient generation time), 0.1 is the decay coefficient, to ensure that the weight decays smoothly with time deviation; For equipment In the j Error at each calibration point This represents the maximum permissible error.

[0043] More preferably, in this embodiment, To the maximum permissible error, when the calibration of all devices is complete. All are not less than the preset threshold (e.g.) When the global calibration is completed, the global calibration model is considered successfully established.

[0044] Based on the generation timestamp of the calibration data, the weight corresponding to the timeliness of the data is calculated using an exponential decay function; this model ensures that data with higher accuracy and more up-to-date timeliness accounts for a larger proportion in the final global calibration model.

[0045] S400. Based on the global calibration model, the coordinates of the subsequently acquired real-time eye-tracking data are corrected to generate eye-tracking coordinates.

[0046] As one embodiment of this application, the specific steps for coordinate correction of the collected eye-tracking data are as follows: For any eye movement data point located at the edge of the screen, based on the spatial distance between the eye movement data point and the preset anchor point of each eye tracker, the calibration completion rate of each eye tracker, and the calibration point density, corresponding optimization weights are generated for each eye tracker. The coordinates of the eye-tracking data points reported by each eye tracker are weighted and averaged using the optimized weights to obtain the corrected coordinates.

[0047] Preferably, coordinate correction is performed on the subsequently acquired real-time eye-tracking data, adding calibration completion rate to the traditional method that relies solely on Euclidean distance. With calibration point density coefficient As a weighting factor. Specifically, the precise coordinates of data points within the overlapping area are analyzed to more accurately determine the region of the pilot's eye movement: First, the physical spatial boundaries of the fixed eye tracker are collected to determine its effective coverage area on the central canvas. Second, feature extraction is performed on real-time eye-tracking data points, with the core parameter being their coordinate information on the calibration canvas. Based on this, the Euclidean distance between the real-time data point and the center calibration anchor point outside the region of each candidate device is calculated. Combined with equipment calibration completion With calibration point density Generate corresponding comprehensive optimization weights Based on this weight, the real-time coordinate values ​​are weighted and optimized to ultimately achieve accurate determination of user eye-tracking data.

[0048] More preferably, coordinate correction is performed on the subsequently acquired real-time eye-tracking data:

[0049] in, This represents the Euclidean distance between the real-time data point and the anchor point of the i-th device. Indicates the real-time data point and the first The Euclidean distance between the anchor points of the equipment. Represents real-time data points With the The Euclidean distance to the nearest calibration point For the first i The coordinates of the center calibration anchor point outside the equipment area. For the first i The device incorporates a comprehensive optimization weight (values ​​ranging from 0 to 1) that considers distance, calibration completion rate, and calibration point density. ， This represents the coordinates of real-time eye-tracking data points after weighted optimization.

[0050] After calibration, the system enters real-time eye-tracking data acquisition mode. The client UI employs a differentiated marking strategy; for example, it dynamically renders real-time eye-tracking point coordinates with green markers and precisely presents the coordinates of eye-tracking points in a "pausing" state with red markers, enabling intuitive differentiation. A categorized data storage mechanism is activated, writing raw real-time eye-tracking point data from different devices into separate device data files. Stored information fields are standardized, including timestamps, data point coordinates, eye-tracking state, device ID, blink status, and left / right eye vector values, and data validation logic is embedded to ensure data integrity, traceability, and high quality.

[0051] Finally, the application system integrates in-depth visualization analysis capabilities without relying on third-party tools such as MATLAB. Users can access all eye-tracking data and generate eye-tracking trajectory clustering maps covering the entire time period using trajectory clustering algorithms (such as K-Means). Simultaneously, users can customize time windows through the interactive interface, and the application system will automatically extract data within that time period, accurately generating corresponding eye-tracking trajectory clustering results and eye-tracking heatmaps, enabling flexible switching from macro to micro analysis.

[0052] All of the above functions are implemented by loading the eye tracker's SDK dynamic library, and the standardized interface shields the differences in underlying hardware. Key data such as the fused standard coefficients and device errors are written to a shared memory pool for later review and retrieval. The entire application system is uniformly managed and controlled through the client layer. Users do not need to manually switch device connection parameters to achieve parallel collaborative work of multiple devices, and communication retry and fault tolerance mechanisms ensure continuous, trouble-free operation in long-term application scenarios such as eye-tracking research for aviators.

[0053] In a preferred embodiment of this application, the method further includes: employing a dynamically adjusted adaptive flow control strategy during the calibration phase; the strategy includes: During the data transmission phase, the communication window is increased to ensure the integrity of data transmission; When the network is congested, the communication window is adjusted according to the priority of data frames to ensure the transmission stability of high-priority data.

[0054] Specifically, a dynamically adjusted adaptive flow control strategy is adopted, including the following: During the data transmission phase, flow control is performed using a sliding window method, and the window size is dynamically adjusted according to the calibration phase. ; in, The size of the data to be transmitted in the current communication window. The preset communication window controls the upper limit of transmitted data. As a weighting factor in the calibration phase, during the formal calibration phase. The values ​​will be increased (e.g., pre-calibration = 0.6, formal calibration = 1.2, data optimization = 0.8, calibration complete = 0.5) to ensure the complete transmission of critical data; For available bandwidth, This represents the average data packet size. When network congestion occurs, the communication window is adjusted according to the priority of data frames to ensure the transmission stability of high-priority data. ; in, For packet loss rate, To adjust the coefficient, The priority compensation coefficient (high priority = 0.9, medium priority = 1.0, low priority = 1.1) reduces the window shrinkage when high priority data is congested, ensuring stable transmission of critical instructions. Meanwhile, for dynamic data such as progress and errors, only the difference is transmitted. When the difference is less than the threshold (like , When a certain condition is met, no data is transmitted, further saving communication bandwidth.

[0055] in, Representative equipment i No. The dynamic data difference of the time, Representative equipment i No. The dynamic data value of the time, Representative equipment i No. ( - 1) The dynamic data value of the time.

[0056] Preferably, in this embodiment, for pilot eye-tracking scenarios in the aviation field, an experimental environment including two commercially available eye trackers was set up, and an experiment was conducted in a simulated flight scenario to carry out multi-device collaborative calibration and eye-tracking data acquisition and analysis experiments. Based on the method of this application, the eye-tracking data of pilots during simulated flight was collaboratively calibrated, acquired and analyzed.

[0057] The specific process is as follows: 1) After the client is started, the operator selects two eye trackers in the client UI and sends a calibration start command. The command is encapsulated as a high-priority frame and transmitted to the server.

[0058] 2) The server sends time reference packets to the two devices via the NTP time synchronization module to ensure synchronization accuracy. Then, through time slice rotation ( Initiate global calibration with the adaptive dwell time strategy.

[0059] 3) During the calibration process, the server transmits the calibration point coordinates via broadcast frames and transmits the equipment calibration progress and error data incrementally. After receiving the data, the client guides the subject to complete the calibration through voice prompts and displays the calibration progress and score in real time.

[0060] 4) When all equipment calibration is complete At that time, real-time eye-tracking data acquisition is initiated, and the data is categorized and stored (including fields such as timestamp, coordinates, and eye-tracking status) and its validity is verified.

[0061] 5) After the data collection is completed, the eye-tracking trajectory clustering map and the heat map of the custom time window are generated directly on the client to complete the in-depth analysis.

[0062] The analytical indicators for the above experiments include the following: Synchronization error data: The local time of the time reference packet received by the two devices and the time sent by the server are recorded through the server's NTP module. The start time deviation is calculated, and the experiment is repeated 30 times and the average value is taken.

[0063] Coordinate error data: Based on the actual coordinates of the calibration point, compare the eye-tracking coordinates collected by the device, calculate the absolute error, and repeat the experiment 30 times to take the average value.

[0064] Communication volume data: Statistics for each calibration are generated through the server network monitoring module. The total amount of data transmitted during the data acquisition process was averaged after repeating the experiment 10 times.

[0065] Packet loss rate data: In a simulated environment with a bandwidth limit of 1Mbps, the percentage of data packets lost during transmission was statistically analyzed, and the average value was taken after 20 repeated experiments.

[0066] Edge data efficiency: Select the canvas edge region (distance from the boundary) Eye-tracking data (pixels) were statistically weighted and optimized to meet the error... The data percentage was calculated by repeating the experiment 30 times and taking the average value.

[0067] Analysis efficiency: The total time taken to generate the full-time cluster map and the 60-second time window heatmap was calculated by repeating the experiment 10 times and taking the average value.

[0068] This embodiment conducted a total of 1200 full-process tests, including 540 tests in a simulator environment (45%) and 660 tests in a laboratory environment (55%). The simulator environment completely replicated the configuration of a civil aviation pilot's and co-pilot's control console, equipped with two eye trackers. The instructor's console system screen resolution was 1920×1080, and the calibration canvas size was fixed at 1600×900 pixels. The laboratory environment tested two eye trackers in parallel, with bandwidth gradually adjusted from 1Mbps to 100Mbps, covering low-bandwidth to high-speed network scenarios. Through repeated experiments and comparison with existing publicly available technical data, the technical effects of this application are quantified as shown in Table 1 below: Table 1

[0069] The test results are as follows: Each test performs a complete nine-point calibration procedure to determine the "calibration completion rate of all devices". "As a success criterion, the number of successful tests was recorded out of 1200 tests. Among them, 528 were successful out of 540 tests in the simulator environment, and 651 were successful out of 660 tests in the laboratory environment, with an overall success rate of approximately 98.3%."

[0070] In 30 specific synchronization tests, the server sent a time reference packet once per second via the NTP module for 5 minutes, recording the difference between the receiving time and the server's sending time for each device. The average synchronization error between the two devices was 0.82ms, with a maximum single deviation of 1.2ms and a minimum deviation of 0.5ms, verifying the stability of the global time synchronization mechanism.

[0071] Using the true normalized coordinates of the calibration point as a benchmark, eye-tracking coordinates of each device were collected, and the absolute error was calculated. In 30 repeated experiments, the overall error range was controlled within 3.6-5.2 pixels, which is 56.7% lower than the error level of 8-12 pixels in existing technologies.

[0072] Eye-tracking data from the canvas edge region (≤100 pixels from the boundary) were selected as test samples. 1000 edge data points were collected in each test. After processing with a weighted optimization algorithm, the percentage of data with a statistical error ≤5 pixels was determined. In 30 tests, the edge data effectiveness rate ranged from a minimum of 83.1% to a maximum of 91.3%, with an average of 87.2%, representing a significant improvement over the 50%-60% effectiveness rate of existing technologies.

[0073] In a scenario where two eye trackers are placed side by side, the total data transfer volume of a single nine-point calibration (approximately 2 minutes) plus 60 seconds of data acquisition was recorded. The data transfer volumes for 10 repeated experiments were 34.2MB, 37.8MB, 35.1MB, 39.4MB, 36.7MB, 33.9MB, 38.2MB, 35.8MB, 37.1MB, and 34.9MB, with an average of 36.5MB. This represents a 63.5% reduction compared to the 80-100MB data transfer volume of existing technologies.

[0074] In a simulated low-bandwidth environment with a bandwidth limit of 1Mbps, 1GB of test data was continuously transmitted, and the number of lost data packets was recorded. In 20 experiments, the packet loss rate ranged from a minimum of 1.7% to a maximum of 2.9%, with an average of 2.3%, verifying the stability of the sliding window flow control and XOR check mechanism in low-bandwidth scenarios.

[0075] All experimental data were verified for reliability using the "triple-repetition verification method": the same test scenario was independently repeated three times by different operators, and if the deviation of the three sets of data exceeded 5%, the test was repeated. In the final 1200 tests, only 18 data were invalid due to equipment hardware failure (accounting for 1.5%), and the remaining data met the deviation requirements, verifying the stability and data reliability of this solution in different scenarios.

[0076] A second embodiment of this application provides a multi-eye tracker data processing device based on shared port communication, used to implement the above method, including: The communication management unit is configured to construct separate logical channels for multiple eye trackers based on the communication data received from multiple eye trackers via a shared communication port. The collaborative scheduling unit is configured to synchronize the time of multiple eye trackers based on a global time reference, and according to the time synchronization results, schedule each eye tracker to feed back calibration data in an orderly manner through its logical channel within different time windows. The model generation unit is configured to determine calibration parameters based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data, so as to establish a global calibration model; The coordinate correction unit is configured to perform coordinate correction on the subsequently acquired real-time eye-tracking data based on the global calibration model to generate eye-tracking coordinates.

[0077] Preferably, the device can be implemented by the server layer in the above-described application system architecture to execute the steps in the aforementioned method embodiments. For clarity, the server layer can be logically divided into a "communication processing submodule," a "core control submodule," and an "SDK interaction submodule," and the functional units of the data processing device described in this embodiment correspond one-to-one with these submodules: The communication management unit is mainly handled by the server's communication processing submodule; it is responsible for listening to the shared communication port, parsing the device identifier and priority information in the incoming data frames, thereby constructing and managing the logical channels with each eye tracker. It also performs low-level communication tasks such as flow control and data verification.

[0078] The collaborative scheduling unit, model generation unit, and coordinate correction unit—these three modules—are all implemented by the server's core control submodule, constituting the core processing logic of this application. The collaborative scheduling unit is responsible for time synchronization based on the global NTP time reference and for executing advanced scheduling strategies such as time slice rotation and adaptive dwell time. The model generation unit is responsible for receiving the calibration data fed back after scheduling and executing an innovative two-dimensional fusion algorithm (combining spatial error and temporal timeliness) to establish a global calibration model. The coordinate correction unit uses the established model to perform precise weighted optimization correction on real-time eye-tracking data, especially edge data.

[0079] The data processing device loads and calls the SDK dynamic libraries of each eye tracker through its SDK interaction submodule, thereby shielding hardware differences and enabling standardized interaction with the hardware-level eye trackers. Simultaneously, the data processing device, through the communication layer, receives commands and provides data feedback to the client layer, forming a complete system closed loop.

[0080] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0081] It should be noted that the multi-eye tracker data processing device based on shared port communication provided in the above embodiments is only an example of the division of the above functional units. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of this application can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of this application are only for distinguishing the various modules or steps and are not considered as an improper limitation of this application.

[0082] A device according to a third embodiment of this application includes: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the above-described multi-eye tracker data processing method based on shared port communication.

[0083] A computer-readable storage medium according to a fourth embodiment of this application stores computer instructions that are executed by a computer to implement the above-described multi-eye tracker data processing method based on shared port communication.

[0084] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the storage device and processing device described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0085] The following is for reference. Figure 4 It shows a schematic diagram of the structure of a computer system for implementing the methods, apparatus, and electronic devices of this application. Figure 4 The server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0086] like Figure 4As shown, the computer system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 302 or programs loaded from storage section 303 into Random Access Memory (RAM) 303. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0087] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0088] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0089] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0091] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0092] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0093] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. Without departing from the principles of this application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of this application.

Claims

1. A method for processing multi-eye tracker data based on shared port communication, characterized in that, Applied to a server, the method includes the following steps: Using a shared communication port, logical channels are constructed for each eye tracker based on the communication data received from multiple eye trackers. Multiple eye trackers are synchronized based on a global time reference, and based on the time synchronization results, each eye tracker is scheduled to feed back calibration data in an orderly manner through its logical channel within different time windows. Based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data, global calibration parameters are determined to establish a global calibration model; Based on the global calibration model, coordinate correction is performed on the subsequently acquired real-time eye-tracking data to generate eye-tracking coordinates.

2. The multi-eye tracker data processing method based on shared port communication according to claim 1, characterized in that, The determination of global calibration parameters includes: Based on the generation timestamp of the calibration data, the weight corresponding to the timeliness of the data is calculated using an exponential decay function; The weights corresponding to spatial calibration errors are determined based on the standard deviation between the coordinates measured by the eye tracker and the actual coordinates of the target calibration point.

3. The multi-eye tracker data processing method based on shared port communication according to claim 1, characterized in that, The collected eye-tracking data was corrected, specifically as follows: For eye-tracking data points located in the preset screen edge area, based on the spatial distance between the eye-tracking data point and the center anchor point of the effective area of ​​each eye tracker, the calibration completion degree of each eye tracker, and the density coefficient of the adjacent calibration points, corresponding optimization weights are generated for each eye tracker. The coordinates of the eye-tracking data points reported by each eye tracker are weighted and averaged using the optimized weights to obtain the corrected coordinates.

4. The multi-eye tracker data processing method based on shared port communication according to claim 1, characterized in that, The construction of their respective logical channels specifically involves: Establish communication connections between the server and multiple eye trackers, parse the device identifiers contained in the received communication data, and divide the shared communication port into multiple logical channels according to the device identifiers, with each device identifier corresponding to a logical channel.

5. A multi-eye tracker data processing method based on shared port communication according to claim 4, characterized in that, The device identifier field contains priority information, and the method further includes: Preemptive transmission scheduling is performed on different types of data based on the priority information to ensure low-latency transmission of critical instructions.

6. The multi-eye tracker data processing method based on shared port communication according to claim 5, characterized in that, The method further includes: The system monitors the data transmission delay of each logical channel in real time, and adaptively increases the priority of the time reference packet transmitted on that logical channel when the delay of any logical channel exceeds a preset threshold.

7. The multi-eye tracker data processing method based on shared port communication according to claim 1, characterized in that, Before scheduling the calibration data from each eye tracker, the following steps are also included: A time-slice round-robin scheduling strategy is adopted, and different start-up times are allocated to each eye tracker according to the device identifier to achieve orderly start-up of collaborative calibration.

8. The multi-eye tracker data processing method based on shared port communication according to claim 1, characterized in that, Before scheduling the calibration data from each eye tracker, the following steps are also included: Based on the historical calibration error of each eye tracker, the dwell time at the calibration point is adaptively adjusted. Common calibration parameters, including calibration point coordinates, are broadcast to all eye trackers at once.

9. A multi-eye tracker data processing method based on shared port communication according to claim 1, characterized in that, The method further includes: during the calibration phase, employing a dynamically adjusted adaptive flow control strategy; the strategy includes: During the data transmission phase, the communication window is increased to ensure the integrity of data transmission; When the network is congested, the communication window is adjusted according to the priority of data frames to ensure the transmission stability of high-priority data.

10. A multi-eye tracker data processing device based on shared port communication, characterized in that, include: The communication management unit is configured to construct separate logical channels for multiple eye trackers based on the communication data received from multiple eye trackers via a shared communication port. The collaborative scheduling unit is configured to synchronize the time of multiple eye trackers based on a global time reference, and according to the time synchronization results, schedule each eye tracker to feed back calibration data in an orderly manner through its logical channel within different time windows. The model generation unit is configured to determine calibration parameters based on the spatial calibration error of each eye tracker and the timeliness of the data in the calibration data, so as to establish a global calibration model; The coordinate correction unit is configured to perform coordinate correction on the subsequently acquired real-time eye-tracking data based on the global calibration model to generate eye-tracking coordinates.

Images