Intelligent glasses system with microsaccade detection function

Abstract

This invention relates to the field of smart glasses technology, specifically disclosing a smart glasses system with micro-saccade detection functionality. The system includes an eye-tracking image acquisition module, a data processing and transmission module, an eye-tracking feature extraction module, a micro-saccade detection and analysis module, an application response and interaction module, and a hardware integration module. The eye-tracking image acquisition module simultaneously captures high-frame-rate eye images using a near-infrared light source and a camera. The data processing and transmission module performs image noise reduction and distortion correction, transmitting the images via a dual-mode link. The eye-tracking feature extraction module locates the pupil center and compensates for head movements, generating feature vectors containing position and velocity. The micro-saccade detection and analysis module identifies real micro-saccades and quantifies parameters using dual-threshold screening and a machine learning model. This invention enables wearable and real-time micro-saccade detection, breaking through the limitations of traditional devices and balancing portability and detection accuracy. It can be used in applications such as attention monitoring and disease detection.

Description

Technical Field

[0001] This invention relates to the field of smart glasses technology, and in particular to a smart glasses system with micro-saccade detection function. Background Technology

[0002] Eye tracking and microsaccade detection are important techniques for studying visual cognition, neuroscience, and clinical diagnosis, such as attention deficit hyperactivity disorder, autism spectrum disorders, and early neurodegenerative diseases. Microsaccades are involuntary, minute eye movements that occur during visual fixation. Their parameters, such as amplitude, velocity, direction, and frequency, are closely related to an individual's cognitive load, attentional state, and specific neurophysiological conditions.

[0003] Microsaccade parameters are important indicators revealing cognitive state and neural function. Current technologies for achieving this function have some significant limitations. High-precision detection devices, such as desktop eye trackers, are bulky and expensive, suitable only for fixed laboratory settings. Portable consumer-grade eye-tracking solutions, such as some VR / AR devices, are generally limited by hardware performance or design goals, making professional microsaccade detection and analysis impossible. Existing technologies suffer from a disconnect between function and form, making it difficult to achieve high-precision, real-time microsaccade detection and analysis within the form of wearable, easily interactive glasses.

[0004] Therefore, there is an urgent need for a smart glasses system with micro-saccade detection function to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to provide a smart glasses system with micro-saccade detection function, comprising:

[0006] An eye-tracking image acquisition module is used to illuminate the user's eyeballs with a miniaturized near-infrared light source integrated inside the frame of smart glasses, and to capture a sequence of reflected light images from the surface of the eyeballs using a high-speed miniature camera that is triggered synchronously with the light source, so as to obtain raw eye-tracking video data containing micro-saccade motion information.

[0007] The data processing and transmission module is used to preprocess the raw eye-tracking video data and transmit the processed eye-tracking image data to the system's computing unit in real time.

[0008] The eye movement feature extraction module is used to receive preprocessed eye movement image data, calculate the two-dimensional coordinates of the pupil center in each frame image through the pupil center localization algorithm and the corneal reflector tracking algorithm, generate an eye movement trajectory sequence that changes over time, and extract eye movement feature vectors containing velocity, acceleration and motion direction feature vectors from the trajectory sequence.

[0009] The micro-saccade detection and analysis module is used to identify and segment micro-saccade events in the eye movement trajectory in real time based on the eye movement feature vector and using a preset micro-saccade discrimination model, and to calculate the amplitude, peak velocity, duration and trigger direction parameters of each micro-saccade event.

[0010] The application response and interaction module is used to execute a predefined response strategy based on the parameter results output by the micro-saccade detection and analysis module, including providing visual feedback on the display screen of the smart glasses, providing auditory cues through bone conduction headphones, and sending detection data and analysis reports to an associated mobile terminal or cloud server via a wireless network.

[0011] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described smart glasses system with micro-saccade detection function.

[0012] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described smart glasses system with micro-saccade detection function.

[0013] The beneficial effects of this application are as follows:

[0014] Firstly, this application integrates a miniaturized near-infrared light source of a specific wavelength and a high-speed micro-camera into the inner side of the eyeglass frame, achieving a hardware foundation for micro-saccade detection comparable to desktop scientific research equipment while maintaining the lightweight and wearable form of eyeglasses. Users can wear and use it naturally during daily activities such as walking and taking transportation, breaking the dependence of high-precision eye movement detection on a fixed laboratory environment.

[0015] Secondly, this application integrates a display screen, such as an optical waveguide lens, into a glasses system with micro-saccade detection capabilities. This serves not only as a data acquisition device but also as an interactive terminal. It can analyze the user's micro-saccade characteristics in real time and provide immediate visual feedback through the display screen or auditory cues through bone conduction headphones, achieving a closed-loop interaction of detection, analysis, and feedback. This greatly expands the application potential of combining micro-saccades with eye tracking. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the hardware integration module in this application.

[0017] Figure 2 This is a schematic diagram of the system structure proposed in one embodiment of this application.

[0018] Explanation of reference numerals in the attached figures:

[0019] 1. Smart glasses frame; 2. Miniaturized near-infrared light source; 3. High-speed miniature camera; 4. Glasses display screen; 5. Temples; 6. Stereo speaker; 7. Computing module; 8. Data cable; 9. Data cable interface.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0022] like Figure 1 As shown, this application provides a smart glasses system with micro-saccade detection function, including:

[0023] The eye-tracking image acquisition module 1 is used to illuminate the user's eyeball with a miniaturized near-infrared light source integrated inside the frame of the smart glasses, and to capture the reflected light image sequence on the surface of the eyeball using a high-speed miniature camera that is triggered synchronously with the light source, so as to obtain high frame rate raw eye-tracking video data containing micro-saccade motion information.

[0024] The data processing and transmission module 2 is used to preprocess the raw eye-tracking video data, including noise reduction, contrast enhancement and image distortion correction, and transmit the processed eye-tracking image data to the system's computing unit in real time via a wired or wireless data link.

[0025] The eye movement feature extraction module 3 is used to receive preprocessed eye movement image data, calculate the two-dimensional coordinates of the pupil center in each frame image through the pupil center localization algorithm and the corneal reflector tracking algorithm, generate an eye movement trajectory sequence that changes over time, and extract an eye movement feature vector containing velocity, acceleration and motion direction feature vectors from the trajectory sequence.

[0026] The micro-saccade detection and analysis module 4 is used to identify and segment micro-saccade events in the eye movement trajectory in real time based on the eye movement feature vector and using a preset micro-saccade discrimination model, and to calculate the amplitude, peak velocity, duration and trigger direction parameters of each micro-saccade event.

[0027] The application response and interaction module 5 is used to execute a predefined response strategy based on the parameter results output by the micro-saccade detection and analysis module, including providing visual feedback on the display screen of the smart glasses, providing auditory cues through bone conduction headphones, and sending detection data and analysis reports to an associated mobile terminal or cloud server via a wireless network.

[0028] As described in modules 1-5 above, this invention achieves real-time capture, accurate analysis, and multi-dimensional interactive response of micro-saccades on smart glasses through the collaborative work of the eye-tracking image acquisition module, data processing and transmission module, eye-tracking feature extraction module, micro-saccade detection and analysis module, and application response and interaction module. This breaks through the scene limitations of traditional micro-saccade detection devices, achieving portable and wearable micro-saccade detection and application, and meeting the needs of large-scale group detection, indoor and outdoor mobile scene use, and daily application for ordinary consumers.

[0029] Microsaccades, as involuntary, minute, and rapid movements of the eye during fixation, have characteristic parameters such as amplitude and peak velocity that are closely related to human attentional state and neurological health. They serve as important evidence for psychological research, early disease screening, and intelligent interaction. Traditional microsaccade detection devices have significant limitations. Desktop devices are bulky and can only be used by a single person in a fixed location, making them unsuitable for large-scale group or mobile scenarios. Portable devices either lack the core capabilities for microsaccade detection or have limited functionality, only capable of data collection without intelligent interaction or feedback mechanisms, making it difficult to apply the detection results immediately. Therefore, there is a need to build an integrated, portable, and fully functional detection system that, while ensuring detection accuracy, achieves a closed-loop process for microsaccade data, from collection, processing, and analysis to application response.

[0030] To address the shortcomings of traditional technologies, the core solution of this invention is to integrate a miniaturized, high-performance eye-tracking acquisition component onto a smart glasses carrier. This is combined with efficient data processing algorithms, precise feature extraction models, and multimodal interaction mechanisms. This not only solves the problem of poor portability of traditional devices but also compensates for the shortcomings of existing wearable devices, such as limited functionality and insufficient detection accuracy, thereby achieving a deep integration of micro-saccade detection and smart glasses functionality.

[0031] The core function of the eye-tracking image acquisition module is to acquire high-quality raw eye-tracking data. It illuminates the user's eyes with a miniaturized near-infrared light source integrated inside the smart glasses frame, using a wavelength of 850 nm or 940 nm. This wavelength is low in irritation to the human eye and clearly shows the pupil outline and corneal reflection, avoiding ambient light interference. A high-speed miniature camera is triggered synchronously with the light source, operating at a frame rate of at least 300 frames per second, capable of completely capturing the rapid movement of micro-saccades. The camera lens optical axis is tilted at 10 to 30 degrees to the normal to the eye surface, ensuring simultaneous capture of a clear pupil outline and corneal reflection. When the user wears the glasses and gazes at a target, the light source operates in a pulsed manner with a preset duty cycle, and the camera simultaneously exposes and captures images, acquiring a sequence of more than 300 frames per second of eye reflection images, forming raw eye-tracking video data containing micro-saccade movement information.

[0032] The data processing and transmission module receives the raw data and optimizes its transmission. The raw image contains issues such as random noise, low contrast between the pupil and surrounding areas, and lens distortion, which can affect the accuracy of subsequent feature extraction. This module first uses a Gaussian spatial filter to smooth the image and reduce random noise interference. Then, it uses an adaptive histogram equalization algorithm to enhance the grayscale contrast between the pupil area and the iris and sclera areas, making the pupil outline clearer. Subsequently, based on pre-calibrated camera lens parameters, it digitally corrects image distortion, restores the true geometric proportions, and converts pixel coordinates to a two-dimensional plane coordinate system with the approximate rotation center of the eyeball as the origin, ensuring the accuracy of the positional data. The processed image data is transmitted to the computing unit in real time via wired or wireless links. When the wireless signal is stable, a low-latency wireless communication protocol is prioritized; when the wireless link quality is poor, it automatically switches to wired transmission to ensure the continuity and real-time nature of data transmission.

[0033] The eye-tracking feature extraction module extracts key motion features from the processed image data. It analyzes the image sequence frame by frame using pupil center localization and corneal reflector tracking algorithms to locate the coordinates of the pupil center and corneal reflector. The relative positional relationship between these two coordinates compensates for the influence of minute head movements, resulting in a clean sequence of pupil center coordinates. This coordinate sequence is then concatenated chronologically to form the original eye-tracking trajectory. A digital low-pass filter removes high-frequency jitter caused by muscle tremors and system noise, resulting in a smooth eye-tracking trajectory sequence. Instantaneous velocity and acceleration of eye movements are calculated using numerical differentiation, and the motion direction angle is obtained by combining this with the direction of movement of the trajectory points. Finally, the pupil position, velocity, acceleration, and motion direction angle are combined and encapsulated to form a multi-dimensional eye-tracking feature vector, providing core data support for micro-saccade detection. For example, when a micro-saccade occurs, rapid positional changes occur in the trajectory sequence within a short period, and the corresponding velocity and acceleration values ​​increase significantly. These features are accurately extracted and incorporated into the feature vector.

[0034] The micro-saccade detection and analysis module is responsible for identifying micro-saccade events and quantifying parameters from eye movement feature vectors. Based on a pre-defined micro-saccade discrimination model, it first scans the velocity data in the feature vectors. When the velocity values ​​of multiple consecutive sample points exceed a first velocity threshold and the duration exceeds a minimum duration threshold, they are marked as candidate micro-saccade events. A machine learning classification model then filters the candidate events, extracting discriminative features such as peak velocity, symmetry of acceleration change curves, and trajectory straightness to eliminate false events caused by interference factors and determine genuine micro-saccades. Finally, the genuine micro-saccade events are quantified by calculating the straight-line distance between the trajectory's start and end points and converting it to visual angle units as the motion amplitude. The maximum value is extracted from the velocity sequence as the peak velocity, the start-end time difference is calculated as the duration, and the trajectory vector direction is determined as the trigger direction, forming a complete micro-saccade parameter set.

[0035] The application response and interaction module enables the immediate application of test results, executing predefined response strategies based on microsaccade parameters. In attention monitoring scenarios, if the frequency or amplitude of microsaccades abnormally decreases or increases, the smart glasses' display screen will show a visual cue icon, and the bone conduction headphones will simultaneously play auditory cues to remind the user to concentrate. In disease detection scenarios, the system sends test data and analysis reports to associated mobile terminals or cloud servers via wireless networks, facilitating subsequent diagnostic assessments by medical personnel. In intelligent interaction scenarios, specific microsaccade patterns can be set as control commands to enable quick operation of the glasses' functions, directly transforming test results into practical value.

[0036] In one embodiment, a hardware integration module 6 is also included, specifically comprising:

[0037] The smart glasses frame 1 is used to support and fix all functional components, and its temples 5 have built-in wiring channels;

[0038] The miniaturized near-infrared light source 2 and the high-speed miniature camera 3 are integrated in the form of a miniaturized module inside the frame near the eyeball, and are connected to the main control board through a flexible circuit.

[0039] The glasses display screen 4 is embedded in front of the lenses of the glasses frame and is used to display visual feedback information;

[0040] The computing module 7 is an independent wearable or portable processing unit that is connected to the smart glasses frame via a wired or wireless data link. It contains a high-performance processor and storage unit for running the eye movement feature extraction module and the micro-saccade detection and analysis module.

[0041] In the aforementioned hardware integration module, the smart glasses frame 1, miniaturized near-infrared light source 2, high-speed miniature camera 3, glasses display screen 4, and computing power module 7 are modularly and miniaturized and integrated, realizing the physical support, precise layout, and efficient collaboration of each functional component, providing stable and reliable hardware support for micro-eye sac detection and intelligent interaction, while taking into account the wearability, portability, and functional integrity of the device.

[0042] Micro-saccade detection requires high-precision eye image acquisition, real-time data processing, and immediate human-computer interaction feedback. The realization of these functions depends on the precise coordination and rational layout of various hardware components. Traditional eye-tracking devices suffer from significant hardware integration deficiencies: desktop-style devices have scattered components, are bulky and impractical for wear; VR glasses are heavy and have a closed front end, making them unsuitable for everyday use; professional eye-tracking glasses lack display components and have limited functionality; and AR glasses lack high-performance eye-tracking detection components and computing units, making micro-saccade detection impossible. Therefore, a compact, lightweight, and functionally compatible hardware integration solution is needed to organically integrate core components such as eye-tracking acquisition, display, and computing power into a smart glasses form factor, meeting both the accuracy requirements and ensuring portability.

[0043] Addressing the hardware integration pain points of traditional technologies, the core solution of this invention is to adopt a modular design approach. After miniaturizing each functional component, it is precisely arranged in the corresponding position of the smart glasses frame according to functional requirements. At the same time, an independent high-performance computing module is set up and connected to the glasses frame through wired or wireless links. While ensuring compact hardware integration, it meets the computing power support required for micro-acroplasty detection, achieving a balance between wearability and functional performance.

[0044] The smart glasses frame 1, serving as the core load-bearing structure, is made of lightweight, high-strength materials, with an overall weight kept below 50 grams to ensure comfort during extended wear. Its temples feature built-in wiring channels, 2 mm wide and 1.5 mm deep, specifically designed to accommodate the flexible circuitry connecting various components, preventing wear or interference caused by exposed wiring while maintaining the device's clean appearance. The frame itself has pre-drilled slots for component installation, precisely matching the dimensions of the light source, camera, and display screen. This ensures that each component is firmly in place after installation, preventing displacement due to head movements and providing a physical basis for stable image acquisition. For example, when the wearer uses the device while walking, the frame's stable structure prevents the camera from shifting relative to the eye, ensuring clear and stable captured images.

[0045] The miniaturized near-infrared light source 2 and the high-speed miniature camera 3 are integrated into a single micro-module. The overall size of the module is controlled within 2.5 mm × 2.5 mm × 3.5 mm (excluding external structures such as the mounting housing other than the camera module). It is fixed inside the frame near the eyeball, about 15 mm away from the eyeball surface, ensuring that the light source can accurately illuminate the eyeball surface, while the high-speed miniature camera 3 can capture clear images of reflected light. Both are connected to the main control board via a flexible circuit. The flexible circuit is only 0.1 mm thick and can deform slightly with the frame without affecting wearing comfort. The miniaturized near-infrared light source 2 uses a near-infrared light-emitting diode with a wavelength of 850 nm, which is small in size and low in power consumption. The camera adopts a global shutter design with a frame rate of up to 600 frames per second, which can completely capture the rapid movement of micro-saccades. This integration method ensures the compactness of the eye-tracking acquisition components and ensures acquisition accuracy through precise layout, solving the problem of poor acquisition effect caused by the scattered components or unreasonable layout of traditional equipment.

[0046] The glasses display screen 4 is embedded in front of the lenses of the frame, parallel to the lenses, and approximately 20 millimeters away from the eyeball. This ensures that the wearer can clearly see the visual feedback information on the display screen while observing the surrounding environment normally. The brightness of the display screen can be adaptively adjusted between 1000 and 3000 nits, a range suitable for both indoor and outdoor use, adapting to the needs of different lighting environments. It features a seamless fit with the frame, with an edge gap of less than 0.5 millimeters, ensuring both the device's airtightness and preventing discomfort caused by the protruding display screen 4. Stereo speakers 6 are also installed on the temples 5, and a data cable interface 9 is located at the end of the temples 5. The data cable interface 9 connects to the computing module via a data cable 8 for power supply and data transmission. For example, during micro-saccade detection, the glasses display screen 4 can display real-time information such as detection progress and attention status prompts, eliminating the need for the wearer to consult other devices and achieving instant detection and feedback.

[0047] The computing module, as an independent wearable processing unit, adopts a cuboid design. Its size can be customized according to actual needs, with specific dimensions of 70 mm × 160 mm × 10 mm. It can be worn around the neck via a lanyard or placed in a pocket. It can be dedicated to running the algorithm programs for the eye-tracking feature extraction module and the micro-saccade detection and analysis module. This solves the problems of insufficient computing power and limited battery life in traditional smart glasses. For example, when processing high frame rate raw eye-tracking video data, the high-performance processor of the computing module can quickly complete feature extraction of a single frame image, ensuring real-time micro-saccade detection.

[0048] In one embodiment, the eye-tracking image acquisition module includes:

[0049] The light source driving and synchronization unit 11 is used to drive the near-infrared light-emitting diode to operate in a pulse mode with a preset duty cycle and generate an electronic trigger signal that is strictly synchronized with each lighting pulse.

[0050] The optical imaging control unit 12 is used to receive the electronic trigger signal and control the high-speed miniature camera to perform exposure and shooting within the corresponding light source illumination pulse duration, and capture an eye image containing a clear pupil outline and corneal reflection spot;

[0051] The image buffer and noise suppression unit 13 is used to process the captured raw image data in real time in the camera or in the adjacent buffer memory, perform fixed-mode noise cancellation and correction of known bad pixels, and acquire raw eye-tracking video data.

[0052] As described in units 11-13 above, through the coordinated operation of the light source driving and synchronization unit, the optical imaging control unit, and the image buffer and noise suppression unit in the eye movement image acquisition module, high frame rate, high definition, and low noise raw eye movement video data acquisition is achieved, accurately capturing eyeball reflection light image sequences containing micro-saccade motion information, providing a reliable data foundation for subsequent data processing, feature extraction, and micro-saccade detection.

[0053] Microsaccades are characterized by their small amplitude and high speed. Their movement amplitude is typically less than 1° of visual field, with peak speeds reaching 30-100 degrees / second and durations of only 10-100 milliseconds. Accurately capturing these rapid and minute movements requires high-frame-rate image acquisition and stable lighting. In existing technologies, traditional desktop eye-tracking devices lack a strict synchronization mechanism between the light source and camera, leading to blurry images or missing frames. Portable devices typically have camera frame rates below 300 frames / second, making it impossible to fully record the microsaccade motion process. Furthermore, the raw images are susceptible to fixed-pattern noise, bad pixels, and ambient light interference, resulting in poor image quality and affecting the accuracy of subsequent analysis. Therefore, a precise acquisition scheme is needed that synchronizes the light source and camera, provides high-frame-rate imaging, and can suppress noise in real time to ensure the integrity and reliability of the raw eye-tracking data.

[0054] Addressing the pain points of existing technologies in image acquisition, the core solution of this invention is to use a near-infrared light source with a specific wavelength for pulsed illumination, control the exposure and shooting of a high frame rate camera through a synchronous triggering mechanism, and simultaneously perform noise suppression and bad pixel correction at the acquisition end to improve image quality from the source and ensure that micro-eye saccade motion information is not missed or interfered with.

[0055] The light source driving and synchronization unit is responsible for providing stable illumination synchronized with the camera. It drives at least one near-infrared LED with a wavelength of 850 nm or 940 nm. These two wavelengths of near-infrared light are low in irritation to the human eye and can effectively penetrate the corneal surface, forming a clear reflected light spot on the eyeball surface while avoiding interference from ambient visible light. The light source operates in pulses with a preset duty cycle of 50%, ensuring sufficient illumination intensity for imaging while reducing device power consumption. Simultaneously with driving the light source, this unit generates an electronic trigger signal that is strictly synchronized with each illumination pulse. The rising edge of the trigger signal is perfectly aligned with the start time of the illumination pulse, ensuring that the camera is exposed only during the illumination period, avoiding uneven image brightness or motion blur caused by asynchronous illumination and shooting. For example, when the light source emits pulses at a frequency of 1000 Hz, the synchronization trigger signal also triggers the camera to capture images at the same frequency, achieving synchronized acquisition every millisecond, ensuring that rapid movements such as micro-saccades are not missed.

[0056] The optical imaging control unit receives the synchronization trigger signal and controls the camera to image. The high-speed miniature camera it controls has a frame rate of no less than 300 frames per second, which is much higher than the movement frequency of micro-saccades, enabling it to completely record the entire process of micro-saccades from initiation, acceleration, peak, to deceleration. The camera's lens optical axis is tilted at an angle of 10 to 30 degrees relative to the normal to the eyeball surface. This angle design allows for the simultaneous capture of a clear pupil outline and corneal reflection spot. The corneal reflection spot is formed by near-infrared light, and its relative position to the pupil center is crucial for subsequent pupil localization and motion compensation. For example, when the lens tilt angle is set to 20 degrees, the camera can clearly capture the circular outline of the pupil and form a stable dot-shaped reflection spot on the corneal surface, providing a clear localization reference for subsequent extraction of eye movement features. Simultaneously, this unit controls the camera's exposure time, setting the exposure duration to 1 microsecond to avoid image blurring caused by excessive exposure, ensuring that the eyeball details in each frame are clear and sharp.

[0057] The image buffer and noise suppression unit is responsible for real-time preprocessing of the raw image data. It processes the captured raw image data within the camera or in an adjacent buffer memory. This unit performs fixed-pattern noise cancellation and correction of known bad pixels. Fixed-pattern noise cancellation uses a row-column correction method, calculating the grayscale difference between each pixel and the pixels in the same row and column to eliminate fixed noise caused by sensor defects. Known bad pixels are recorded in a lookup table during factory calibration. During processing, the grayscale value of the bad pixels is replaced by the average grayscale value of adjacent pixels. For example, when the camera captures three bad pixels in a frame, this unit immediately calls the lookup table to locate the bad pixels and replaces them with the average grayscale value of the eight surrounding pixels. Simultaneously, it eliminates fixed stripe noise from the sensor, improving the contrast between the pupil and background in the image by 30%, effectively reducing the data load and processing pressure of subsequent transmission links.

[0058] In one embodiment, the data processing and transmission module includes:

[0059] The front-end image optimization unit 21 is used to perform sequential processing on each frame of the original image and apply an adaptive histogram equalization algorithm to enhance the grayscale contrast between the pupil region and the surrounding iris and sclera regions in the image.

[0060] The optical distortion correction and coordinate system transformation unit 22 is used to digitally correct the shape distortion caused by the lens characteristics in the image according to the pre-calibrated camera lens parameters, restore the true geometric proportions of the scene, and convert the corrected image pixel coordinates into coordinates in a two-dimensional plane coordinate system with the approximate rotation center of the eyeball as the origin, according to the calibration model.

[0061] As described in units 21-22 above, through the collaborative work of the front-end image optimization unit and the optical distortion correction and coordinate system transformation unit in the data processing and transmission module, the raw eye-tracking video data acquired by the eye-tracking image acquisition module is subjected to noise reduction, contrast enhancement, and distortion correction, and coordinate system transformation is completed to output high-quality, geometrically accurate, and coordinate-standardized eye-tracking image data. This provides accurate data support for subsequent eye-tracking feature extraction and micro-saccade detection, ensuring the reliability of the detection results.

[0062] The raw images acquired by the eye-tracking image acquisition module are affected by environmental noise, sensor performance, and lens optical characteristics. Random electronic noise in the images causes blurred pupil edges and low grayscale contrast between the pupil, iris, and sclera, making it difficult to accurately locate the pupil center. Radial and tangential distortions of the camera lens lead to geometric distortion, causing inconsistencies between the actual pupil shape and the imaged shape. The lack of a unified reference for pixel coordinates prevents a direct reflection of the true eye movement trajectory. These issues directly increase subsequent pupil localization errors, lead to inaccurate eye movement feature extraction, and ultimately affect the accuracy of micro-saccade detection. Therefore, targeted preprocessing of the raw images is necessary to eliminate noise and distortion, unify coordinate references, and improve image data quality.

[0063] To address the shortcomings of existing preprocessing technologies, the core solution of this invention is to integrate a lightweight optimization algorithm into the local processor of smart glasses. First, Gaussian filtering and adaptive histogram equalization are used to improve image quality. Then, pre-calibrated lens parameters are used to correct distortion and convert pixel coordinates into standardized coordinates with the eyeball rotation center as the origin. This maximizes data accuracy while ensuring processing efficiency and adapts to the computing power limitations of wearable devices.

[0064] The front-end image optimization unit is integrated into the processor of the smart glasses and adopts a lightweight algorithm architecture to adapt to the limited computing resources of the glasses. This unit processes each frame of the original image sequentially. First, it uses a Gaussian spatial filter for smoothing, with a kernel size of 5×5. The weight coefficients within the kernel are distributed according to a Gaussian distribution, with the highest weight for the center pixel and gradually decreasing towards the edges. The weighted average of the pixels within the kernel is used to replace the grayscale value of the center pixel, effectively reducing random noise caused by sensor electronic noise and ambient light interference. Then, an adaptive histogram equalization algorithm is applied, dividing the image into 8×8 sub-regions. Each sub-region independently calculates its histogram and performs equalization processing. Finally, bilinear interpolation is used to fuse the grayscale values ​​of adjacent sub-regions, avoiding the local overexposure problem caused by traditional global histogram equalization and significantly enhancing the grayscale contrast between the pupil area and the surrounding iris and sclera areas. For example, when the grayscale difference between the pupil and iris in the original image is only 20, the difference can be increased to 60 after processing, and the clarity of the pupil outline edge is improved by 40%, providing a clear recognition target for subsequent pupil localization algorithms.

[0065] The optical distortion correction and coordinate system transformation unit is responsible for correcting image geometric distortion and unifying the coordinate reference. The camera lens parameters used by this unit are obtained through a pre-shipment calibration process using the Zhang Zhengyou calibration method. This involves capturing multiple sets of checkerboard images in different poses to calculate the lens's intrinsic parameters (including focal length and principal point coordinates) and distortion coefficients (including radial and tangential distortion coefficients), which are then stored in the parameter memory on the glasses. During processing, this unit calls upon the stored calibration parameters and corrects the coordinates of each pixel in the image using a distortion correction model, eliminating radial and tangential distortion caused by the lens and restoring the true geometric proportions of the scene. For example, an elliptical pupil image caused by radial distortion before correction is restored to a standard circle after correction, with the center position deviation controlled within one pixel. Subsequently, based on the preset calibration model, the corrected image pixel coordinates are converted into coordinates in a two-dimensional plane coordinate system with the approximate rotation center of the eyeball as the origin. The calibration model is established by measuring the relative positional relationship between the rotation center of the eyeball and the imaging plane of the camera. The converted coordinates directly reflect the real positional change of the pupil relative to the rotation center of the eyeball, providing a unified and accurate positional reference for subsequent calculation of eyeball movement trajectory and micro-saccade parameters.

[0066] In one embodiment, the data processing and transmission module further includes:

[0067] The data stream organization and encapsulation unit 23 is used to pack multiple consecutive frames of image data after correction and coordinate transformation, or feature data initially extracted from the image, in chronological order, and add a timestamp and a unique incrementing sequence identifier to each data packet.

[0068] The adaptive dual-mode link management unit 24 is used to dynamically manage the data transmission path from the glasses to the computing unit. When the wireless signal strength and signal-to-noise ratio are both higher than the preset threshold, it prioritizes sending data through the low-latency wireless communication protocol. When the end-to-end delay of wireless transmission is detected to exceed the maximum allowable value or the continuous packet loss rate rises to the warning threshold, it automatically and seamlessly switches to the wired transmission channel composed of the physical data interface and connection line for data transmission. After the wireless link quality recovers to a stable state, it automatically switches back to the wireless transmission mode.

[0069] As described in units 23-24 above, through the collaborative work of the data flow organization and encapsulation unit and the adaptive dual-mode link management unit in the data processing and transmission module, the eye-tracking data after correction and coordinate transformation is organized in an orderly and standardized manner and encapsulated. The wired and wireless transmission links are dynamically switched to achieve efficient, continuous and reliable data transmission from the glasses to the computing unit, which ensures the real-time performance of subsequent eye-tracking feature extraction and micro-saccade detection, and adapts to the complex and ever-changing usage scenarios of wearable devices.

[0070] Eye-tracking data, after optical distortion correction and coordinate system transformation, consists of continuous multi-frame image or feature data. Disordered transmission of this data can lead to chaotic parsing by the computing module, making it impossible to accurately reconstruct the eye-tracking trajectory. Furthermore, as wearable devices, smart glasses are used in various scenarios, including indoors and outdoors, and in mobile situations. A single transmission link is susceptible to environmental interference—wireless links experience increased latency and packet loss rates under obstruction or electromagnetic interference, while wired links, though stable, lack portability. Traditional wearable devices often employ a single transmission mode, failing to balance portability and stability. Data transmission lacks standardized encapsulation, simply splicing data without timestamps or sequence identifiers, leading to data misalignment or loss and subsequent processing errors. Transmission link switching relies on manual operation, making it unsuitable for dynamic changes in scenarios. Therefore, a solution for orderly data encapsulation and dynamic link management is needed to ensure the orderliness, real-time nature, and reliability of data transmission.

[0071] Addressing the transmission pain points of existing technologies, the core solution of this invention is to standardize and encapsulate data in chronological order, add a unique identifier and time reference, and monitor the quality of the wireless link in real time. Based on a preset threshold, it automatically switches to wired transmission, ensuring orderly data parsing while taking into account the portability and stability of transmission, thus adapting to the usage needs of wearable devices.

[0072] The data stream organization and packaging unit receives processed image data from the optical distortion correction and coordinate system transformation unit, or feature data such as pupil coordinates initially extracted from the front end, and packages them according to the time sequence of data generation. Each data packet is given a timestamp accurate to milliseconds and a unique incrementing sequence identifier. The timestamp is generated based on the system UTC clock of the smart glasses and is synchronized with the synchronization trigger signal of eye-tracking image acquisition, ensuring that the computing module can reconstruct the eye-tracking process in chronological order. The sequence identifier starts from 1 and increments in the order of data packet generation, facilitating the computing module to verify the integrity of the data packets and request retransmission if missing data is found. For example, when processing the 40th frame of data, this unit packages the data from the 21st to the 40th frames, adds the timestamp "1699999999876" and the sequence identifier "2", forming a standardized data packet to ensure that the data is not out of order during transmission.

[0073] The adaptive dual-mode link management unit dynamically manages the data transmission path. Its preset wireless link thresholds are: wireless signal strength ≥ -60dBm and signal-to-noise ratio ≥ 25dB. When these conditions are met, low-latency wireless communication protocols are prioritized for data transmission. This unit monitors parameters such as end-to-end latency and continuous packet loss rate in real time. When the end-to-end latency exceeds the maximum allowable value of 50 milliseconds, or the continuous packet loss rate rises to the warning threshold of 5%, a seamless switching mechanism is immediately initiated. Within 50 milliseconds, it switches to a wired transmission channel consisting of a physical data interface and connecting cable. Wired transmission uses a USB-C interface with a transmission rate of no less than 10Gbps to ensure uninterrupted data transmission. Once the wireless link quality recovers to the preset threshold and stabilizes for 3 seconds, it automatically switches back to wireless transmission mode, balancing portability in mobile scenarios with stability in complex environments. For example, if a user wearing glasses moves from an open indoor space into an elevator, and the wireless signal strength drops to -75dBm and the continuous packet loss rate rises to 8%, the unit immediately switches to wired transmission, ensuring uninterrupted data transmission within the elevator. After exiting the elevator, wireless transmission resumes, and the unit automatically switches back to wireless mode.

[0074] In one embodiment, the eye-tracking feature extraction module includes:

[0075] The pupil positioning and compensation unit 31 is used to analyze the received image sequence frame by frame, locate the center coordinates of the pupil and the coordinates of the corneal reflection point generated by the light source in each frame image, and use the relative positional relationship between the reflection point and the pupil center to calculate and compensate the pupil center coordinates in real time to obtain a pure pupil center coordinate sequence in the eye coordinate system.

[0076] The trajectory generation and smoothing unit 32 is used to connect the pure pupil center coordinates in chronological order to form the original eye movement trajectory, use a digital low-pass filter to filter the original eye movement trajectory, and output a smooth eye movement trajectory sequence.

[0077] The kinematic feature calculation unit 33 is used to perform numerical differentiation operations on the smooth eye movement trajectory sequence, calculate the instantaneous velocity value and instantaneous acceleration value of the eye movement, and calculate the eye movement direction angle at each sampling moment according to the movement direction of the trajectory point;

[0078] The feature vector generation and output unit 34 is used to combine and encapsulate the pupil position, instantaneous velocity, instantaneous acceleration and motion direction angle corresponding to each sampling moment to form a multi-dimensional eye movement feature vector arranged in chronological order, and output the eye movement feature vector to the micro-saccade detection and analysis module.

[0079] As described in units 31-34 above, through the collaborative work of the pupil localization and compensation unit, trajectory generation and smoothing unit, kinematic feature calculation unit and feature vector generation and output unit in the eye movement feature extraction module, multi-dimensional features such as the position, velocity, acceleration and direction of pupil movement are accurately extracted from the preprocessed eye movement image data, and standardized eye movement feature vectors are generated. This provides accurate and comprehensive input data for the micro-saccade detection and analysis module, ensuring the accuracy of micro-saccade event recognition and parameter calculation.

[0080] The core characteristics of microsaccades lie in their kinematic parameters. Their velocity, acceleration, and direction of motion are key differentiators from other eye movements (such as drift and tremor). However, actual eye movement data is subject to interference. Minor head movements while wearing smart glasses can cause pupillary imaging position shifts, obscuring the true eye movements. The original eye movement trajectory includes high-frequency jitter caused by muscle tremors and system noise, masking the true trajectory of microsaccades. Relying solely on pupillary position data cannot reflect the dynamic characteristics of microsaccades and is insufficient to meet the discrimination requirements of detection algorithms. Traditional eye movement feature extraction does not compensate for minor head movements, resulting in large positioning errors. Trajectory smoothing uses simple averaging filtering, easily losing the rapid movement details of microsaccades. Extracting only positional features lacks key kinematic parameters such as velocity and acceleration, leading to a high misclassification rate in subsequent detection. Therefore, a complete extraction scheme including positioning compensation, trajectory optimization, and multi-dimensional feature calculation is needed to ensure that the feature data accurately reflects the essential characteristics of microsaccades.

[0081] Addressing the pain points of feature extraction in existing technologies, the core solution of this invention is to first compensate for head movement by the relative positional relationship between the pupil and the corneal reflection point, then perform targeted smoothing on the trajectory, and finally calculate multi-dimensional kinematic features and encapsulate them into standardized vectors. While eliminating interference, it fully preserves the key features of micro-saccades, thus meeting the needs of micro-saccade detection algorithms.

[0082] The pupil localization and compensation unit receives the preprocessed image sequence output by the data processing and transmission module and analyzes the pupil and corneal reflection points in the image frame by frame. This unit uses threshold segmentation combined with the Hough circle detection algorithm to locate the pupil center coordinates. First, an adaptive threshold is used to divide the image into foreground (pupil) and background (iris, sclera). Then, the circular outline of the pupil is identified using Hough circle detection, and the center of the outline is calculated as the pupil center coordinates. The corneal reflection points are formed by near-infrared light source illumination and are located using a template matching algorithm. A preset grayscale template of reflection points is used to traverse the image, and the region with the highest similarity is matched as the reflection point coordinates. Since the relative position of the corneal reflection point and the pupil center remains stable during slight head movements, this unit uses the corneal reflection point as a reference to calculate the offset of the pupil center relative to the reflection point. This offset is subtracted from the original pupil center coordinates to obtain a pure pupil center coordinate sequence that eliminates the influence of head movement. For example, a slight head movement results in the original coordinates of the pupil center being (100, 80), the coordinates of the corneal reflection point being (95, 75), and the relative offset being (5, 5) in a certain frame of the image. After compensation, the pure coordinates are (95, 75), ensuring that the coordinates can truly reflect the movement of the eyeball itself.

[0083] The trajectory generation and smoothing unit connects the pure pupil center coordinates in chronological order to form the original eye movement trajectory. The original trajectory contains muscle tremors (frequency approximately 20-30Hz) and high-frequency jitter caused by system noise. If not processed, it will lead to distortion in subsequent kinematic parameter calculations. This unit uses a Butterworth digital low-pass filter for filtering, with a cutoff frequency of 5Hz. This frequency can filter out high-frequency jitter while completely preserving the movement trajectory of micro-saccades (frequency approximately 1-4Hz). During the filtering process, a recursive algorithm is used to perform convolution operations on each coordinate point of the original trajectory, outputting a smooth eye movement trajectory sequence. For example, if a segment in the original trajectory experiences coordinate fluctuations due to muscle tremors (jumping from (95,75) to (97,77) and then back to (95,75)), after filtering, the trajectory smoothly transitions to (95,75) → (96,76) → (95,75), eliminating jitter without losing the true movement trend.

[0084] The kinematic feature calculation unit performs numerical differentiation on the smooth eye movement trajectory sequence to calculate the instantaneous velocity and acceleration values ​​of the eye movements. Based on a camera frame rate of 600 frames / second and a time interval of 1 / 600 second, the first-order central difference method is used to calculate the instantaneous velocity, i.e., the velocity at a certain moment is equal to the position difference between two adjacent moments divided by twice the time interval. The second-order central difference method is used to calculate the instantaneous acceleration, i.e., the acceleration at a certain moment is equal to the velocity difference between two adjacent moments divided by twice the time interval. The motion direction angle is calculated using the arctangent function, with 0 degrees to the right horizontally. Based on the position offset (Δx, Δy) between the current moment and the previous moment, the direction angle θ = arctan(Δy / Δx) is calculated, ranging from 0 to 360 degrees. For example, if the coordinates of a smooth trajectory at a certain moment are (95, 75), the previous moment is (90, 75), and the next moment is (100, 75), then the instantaneous velocity is ((100-90) / (2×(1 / 600))) = 3000 pixels / second, the instantaneous acceleration is 0, and the motion direction angle is 0 degrees, accurately reflecting the uniform motion state of the eyeball horizontally to the right.

[0085] The feature vector generation and output unit combines and encapsulates the pupil position (x, y), instantaneous velocity (vx, vy), instantaneous acceleration (ax, ay), and motion direction angle (θ) corresponding to each sampling moment to form an 8-dimensional eye-tracking feature vector. The vector format is (x, y, vx, vy, ax, ay, θ, t), where t is the timestamp of that moment (synchronized with the timestamp during data encapsulation). The feature vectors are arranged in chronological order and output to the micro-saccade detection and analysis module in real time, providing comprehensive and standardized feature input for the recognition of micro-saccade events.

[0086] In one embodiment, the microsaccade detection and analysis module includes:

[0087] The preliminary event detection unit 41 is used to scan the eye movement feature vector and identify possible micro-saccade intervals according to preset judgment rules. When the eye movement velocity values ​​of several consecutive data sample points exceed the first velocity threshold and the duration of the high-speed state exceeds the minimum duration threshold, the time interval is marked as a candidate micro-saccade event and its start and end times are recorded.

[0088] The refined classification and screening unit 42 is used to perform in-depth analysis on each candidate event to eliminate false events. It extracts a set of identification features from the data segment of the candidate event, including the peak velocity of the event, the symmetry of the acceleration change curve, and the overall straightness of the motion trajectory. The identification features are input into a pre-trained machine learning classification model, which determines whether the candidate event is a real micro-saccade based on the learned pattern.

[0089] The event parameter quantization unit 43 is used to accurately measure events that are determined to be real micro-saccades by the classification model, calculate the motion amplitude of the event, that is, the straight-line distance between the starting point and the ending point of the trajectory on the two-dimensional plane, convert it into a visual unit, calculate the difference between the end time and the start time of the event as the duration, and calculate the vector direction from the starting point to the ending point as the main direction of the micro-saccade.

[0090] As described in units 41-43 above, through the collaborative work of the preliminary event detection unit, the refined classification and screening unit, and the event parameter quantification unit in the micro-saccade detection and analysis module, real micro-saccade events are accurately identified from the eye movement feature vector, spurious event interference is eliminated, and the core parameters of micro-saccades are quantified and output, providing accurate and reliable decision-making basis for the application response and interaction module, and ensuring the effectiveness of the system in scenarios such as attention monitoring and disease detection.

[0091] The motion characteristics of micro-saccades partially overlap with other eye movement forms such as eye drift and muscle tremors. Furthermore, eye movement feature vectors may contain spurious events caused by slight head movements or sensor noise. If these spurious events cannot be effectively distinguished, the misclassification rate of micro-saccade detection will be high, impacting subsequent application decisions. Traditional micro-saccade detection relies solely on a single velocity threshold, neglecting multi-dimensional features such as duration and motion trajectory. This easily leads to misclassifying rapid drifts as micro-saccades, lacks a refined screening mechanism, fails to eliminate spurious events, and parameter calculations are limited to simple numerical statistics without conversion to standardized units (such as viewing angle), making data comparison between different devices impossible. Therefore, a three-tiered detection process—preliminary screening, refined identification, and precise quantification—is needed to ensure the accuracy of micro-saccade event recognition and the standardization of parameters.

[0092] Addressing the pain points of existing detection technologies, the core solution of this invention is to first initially identify candidate events by using dual thresholds for speed and duration, then extract multidimensional discriminative features and use machine learning models to screen for real micro-saccades, and finally quantify the core parameters using standardized methods. While eliminating interference, it outputs detection results with practical application value, adapting to the usage needs of different scenarios.

[0093] The preliminary event detection unit receives eye movement feature vectors output by the eye movement feature extraction module. These vectors contain multi-dimensional data such as pupil position, instantaneous velocity, instantaneous acceleration, and motion direction angle. The core parameters of the unit's preset judgment rules are: a first velocity threshold of 20 degrees / second, based on the physiological characteristic that the peak velocity of microsaccades is typically between 30-100 degrees / second; eye movements below this threshold can be judged as non-microsaccades. A minimum duration threshold of 10 milliseconds corresponds to the physiological range of 10-100 milliseconds for microsaccade duration; high-speed movements shorter than this duration are often noise interference. The unit scans the instantaneous velocity data in the eye movement feature vector in chronological order. When the velocity values ​​of three or more consecutive data points exceed 20 degrees / second, and the duration of this high-speed state exceeds 10 milliseconds, the unit immediately marks this time interval as a candidate microsaccade event and records its start and end times. For example, in a certain feature vector, the velocity values ​​from millisecond 100 to 108 are all higher than 20 degrees / second, lasting for 8 milliseconds, which does not reach the threshold and is not marked; the velocity values ​​from millisecond 150 to 165 are all higher than 20 degrees / second, lasting for 15 milliseconds, which meets the condition and is marked as a candidate event.

[0094] The refined classification and screening unit performs in-depth identification on each candidate event, eliminating pseudo-events. This unit extracts three core identification features from the data segment corresponding to the candidate event: peak velocity is the maximum value in the event's velocity sequence; the symmetry of the acceleration change curve is calculated by the difference in slope between the rising edge (from start to peak velocity) and the falling edge (from peak velocity to stop), with a smaller difference indicating better symmetry; and the overall straightness of the trajectory is calculated by the average distance from all points on the trajectory to the fitted straight line, with a smaller distance indicating higher straightness. These features effectively distinguish between genuine saccades and pseudo-events. Genuine saccades typically have peak velocities higher than 30 degrees / second, good acceleration curve symmetry, and high trajectory straightness, while pseudo-events often exhibit lower peak velocities, irregular acceleration curves, and scattered trajectories. This unit inputs the three sets of identification features into a pre-trained support vector machine learning classification model. The model, through the feature patterns of genuine saccades and pseudo-events learned during the training phase, outputs classification results, determining whether the candidate event is a genuine saccade or an artifact caused by interference. The training set of the support vector machine model contains 10,000 samples, including 6,000 real micro-saccade samples and 4,000 pseudo-event samples (including interference types such as head shaking and sensor noise). After training, the model's classification accuracy reaches over 98%, effectively eliminating various pseudo-events.

[0095] The event parameter quantization unit performs precise parameter calculations for events identified as genuine micro-saccades by the classification model. The motion amplitude calculation first obtains the straight-line distance between the start and end points of the event trajectory in a two-dimensional plane coordinate system. Then, it converts this distance to angle of view units based on camera calibration parameters (the conversion ratio between pixels and viewing angle). The conversion ratio is determined through pre-shipment calibration; for example, 10 pixels correspond to 0.1 degrees of viewing angle. If the straight-line distance is 30 pixels, the amplitude is 0.3 degrees of viewing angle. The peak velocity is directly extracted from the maximum value of the velocity sequence during the event's duration. The duration is obtained by calculating the difference between the event's end and start times. The trigger direction is determined by calculating the vector direction angle from the trajectory's start point to its end point, with 0 degrees to the right horizontally and increasing counter-clockwise, ranging from 0 to 360 degrees. For example, the trajectory of a real micro-saccade event has the starting coordinates (100,80) and the ending coordinates (103,80), with a straight-line distance of 30 pixels, a converted amplitude of 0.3 degrees of viewing angle, a maximum velocity sequence value of 45 degrees / second, a start time of 150 milliseconds and an end time of 165 milliseconds, a duration of 15 milliseconds, a vector direction of horizontal to the right, and a trigger direction angle of 0 degrees.

[0096] In one embodiment, the adaptive dual-mode link management unit further includes a data transmission optimization strategy, specifically including:

[0097] The bandwidth dynamic allocation subunit 241 is used to dynamically adjust the bandwidth allocation of the wireless channel according to the data type currently being transmitted. When transmitting compressed original image sequences, high bandwidth is allocated to ensure data integrity. When transmitting only lightweight feature data such as extracted pupil coordinates, bandwidth usage is automatically reduced to save power consumption.

[0098] The forward error correction and retransmission control subunit 242 is used to apply forward error correction coding to the encapsulated data packets in wireless transmission mode, and perform decoding and error correction at the receiving end to establish an acknowledgment and retransmission mechanism based on sequence identifiers. It selectively retransmits only data packets that are not successfully received or fail to be verified, so as to balance the requirements of transmission reliability and real-time performance.

[0099] The link health assessment subunit 243 is used to continuously monitor the signal strength, signal-to-noise ratio, historical packet loss rate and latency jitter of the wireless link, and calculate a real-time link health score based on these indicators, which serves as the main decision-making basis for whether to trigger wired or wireless mode switching.

[0100] As described in units 241-243 above, through the collaborative work of the bandwidth dynamic allocation subunit, the forward error correction and retransmission control subunit, and the link health assessment subunit in the adaptive dual-mode link management unit, the resource allocation, reliability, and switching decisions of wireless transmission are optimized. Under the premise of ensuring data transmission integrity and real-time performance, system power consumption is reduced, providing a scientific basis for accurate switching of dual-mode links, and further improving the stability and adaptability of data transmission in the entire system.

[0101] Wireless transmission is the primary data channel between smart glasses and the computing module. However, different data types (such as raw image sequences and lightweight feature data) vary greatly in size and bandwidth requirements. The wireless environment is susceptible to obstruction and electromagnetic interference, leading to data packet loss. A single link indicator (such as signal strength alone) cannot fully reflect the true state of the link, easily causing erroneous switching. Traditional wearable devices use fixed bandwidth allocation, resulting in bandwidth waste when transmitting feature data and latency due to insufficient bandwidth when transmitting raw images. Wireless transmission relies solely on a simple retransmission mechanism without error correction coding, leading to low retransmission efficiency after packet loss and increased latency. Link switching is based on a single indicator, resulting in biased decision-making and erroneous switching to wired mode during temporary link fluctuations, affecting portability. Therefore, a targeted transmission optimization strategy is needed to achieve on-demand bandwidth allocation, intelligent packet loss error correction, and accurate link status assessment, comprehensively improving wireless transmission performance.

[0102] Addressing the pain points of transmission optimization in existing technologies, the core solution of this invention is to dynamically adjust bandwidth allocation based on data type, reduce retransmissions through forward error correction coding, and comprehensively evaluate link health using multi-dimensional indicators. This ensures transmission quality while reducing power consumption, avoiding accidental handovers, and fully leveraging the portability advantages of wireless transmission.

[0103] The bandwidth dynamic allocation subunit identifies the data type being transmitted in real time. Data types are distinguished by preset identifiers in the packet header: "01" represents compressed raw image sequences, and "02" represents lightweight feature data such as pupil coordinates. This unit has preset bandwidth allocation rules: when transmitting raw image sequences, 80% of the wireless channel bandwidth is allocated to meet the bandwidth requirements of large data transmissions; when transmitting lightweight feature data, bandwidth usage is automatically reduced to 20%, reducing the power consumption of the wireless module. For example, when transmitting a single frame of 1MB raw image data, the 80% bandwidth allocation allows the actual transmission rate of the Wi-Fi 6 link to reach 1.92Gbps, with a single frame transmission latency controlled within 5 milliseconds. When transmitting a single frame of 2KB pupil coordinate data, a 20% bandwidth allocation is sufficient, reducing the wireless module power consumption from 1.2W to 0.8W and extending the device's battery life by 30%.

[0104] The forward error correction and retransmission control subunit is optimized to address the instability of wireless transmission. In wireless transmission mode, RS (Reed-Solomon) forward error correction coding is applied to the data packets generated by the data stream organization and encapsulation unit, with a coding rate set to 3 / 4, meaning one check block is added for every four data blocks. The receiver can directly correct errors in a single data block using the check block, eliminating the need for retransmission. Simultaneously, an acknowledgment and retransmission mechanism based on sequence identifiers is established. After receiving every 10 data packets, the receiver sends an acknowledgment signal to the glasses, clearly marking the sequence number of the data packets that were not successfully received or failed verification. The glasses selectively retransmit only the marked data packets, rather than retransmitting the entire data segment, significantly reducing latency caused by retransmission. For example, if the 7th data packet in a batch of 10 fails verification due to electromagnetic interference, after the receiver's feedback, the glasses only retransmit the 7th data packet, keeping the retransmission latency within 20 milliseconds, an 80% reduction compared to full retransmission.

[0105] The link health assessment subunit continuously monitors four core indicators of the wireless link: signal strength, signal-to-noise ratio (SNR), historical packet loss rate, and latency jitter. Each indicator contributes to the score with equal weight (25%), and the total score ranges from 0 to 100. Signal strength ≥ -60dBm scores 25 points, -70 to -60dBm scores 15 points, and < -70dBm scores 5 points; SNR ≥ 25dB scores 25 points, 15 to 25dB scores 15 points, and < 15dB scores 5 points; historical packet loss rate < 1% scores 25 points, 1% to 5% scores 15 points, and > 5% scores 5 points; latency jitter < 10ms scores 25 points, 10 to 30ms scores 15 points, and > 30ms scores 5 points. A comprehensive score ≥ 80 indicates a healthy link, 60 to 79 indicates a fair link, and < 60 indicates a poor link. This score directly serves as the basis for dual-mode switching decisions. For example, if the detected signal strength is -55dBm (25 points), signal-to-noise ratio is 30dB (25 points), historical packet loss rate is 0.5% (25 points), and latency jitter is 8 milliseconds (25 points), the overall score is 100 points, the link is determined to be healthy, and wireless transmission is maintained. If the detected signal strength is -75dBm (5 points), signal-to-noise ratio is 12dB (5 points), historical packet loss rate is 6% (5 points), and latency jitter is 35 milliseconds (5 points), the overall score is 20 points, the link is determined to be poor, and wired handover is triggered.

[0106] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described smart glasses system with micro-saccade detection function.

[0107] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described smart glasses system with micro-saccade detection function.

[0108] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in this application and in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0109] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0110] The above description is merely a preferred embodiment of the present invention and does not limit the scope of this application. Any equivalent results or equivalent process transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of this application.

Claims

1. A smart glasses system with microsaccade detection functionality, characterized in that, include: An eye-tracking image acquisition module is used to illuminate the user's eyeballs with a miniaturized near-infrared light source integrated inside the frame of smart glasses, and to capture a sequence of reflected light images from the surface of the eyeballs using a high-speed miniature camera that is triggered synchronously with the light source, so as to obtain raw eye-tracking video data containing micro-saccade motion information. The data processing and transmission module is used to preprocess the raw eye-tracking video data and transmit the processed eye-tracking image data to the system's computing unit in real time. The eye movement feature extraction module is used to receive preprocessed eye movement image data, calculate the two-dimensional coordinates of the pupil center in each frame image through the pupil center localization algorithm and the corneal reflector tracking algorithm, generate an eye movement trajectory sequence that changes over time, and extract eye movement feature vectors containing velocity, acceleration and motion direction feature vectors from the trajectory sequence. The micro-saccade detection and analysis module is used to identify and segment micro-saccade events in the eye movement trajectory in real time based on the eye movement feature vector and using a preset micro-saccade discrimination model, and to calculate the amplitude, peak velocity, duration and trigger direction parameters of each micro-saccade event. The micro-saccade detection and analysis module includes: The preliminary event detection unit is used to scan the eye movement feature vector and identify possible micro-saccade intervals according to preset judgment rules. When the eye movement velocity values ​​of several consecutive data sample points exceed the first velocity threshold and the duration of the velocity state exceeds the minimum duration threshold, the time interval is marked as a candidate micro-saccade event and its start and end times are recorded. The refined classification and screening unit is used to perform in-depth analysis on each candidate event to eliminate false events. It extracts a set of identification features from the data segment of the candidate event, including the peak velocity of the event, the symmetry of the acceleration change curve, and the overall straightness of the motion trajectory. The identification features are input into a pre-trained machine learning classification model, which determines whether the candidate event is a real micro-saccade based on the learned pattern. The event parameter quantization unit is used to accurately measure events that are identified as real micro-saccades by the classification model, calculate the motion amplitude of the event, that is, the straight-line distance between the starting point and the ending point of the trajectory on the two-dimensional plane, convert it into a visual unit, calculate the difference between the end time and the start time of the event as the duration, and calculate the vector direction from the starting point to the ending point as the main direction of the micro-saccade. The application response and interaction module is used to execute a predefined response strategy based on the parameter results output by the micro-saccade detection and analysis module, including providing visual feedback on the display screen of the smart glasses, providing auditory cues through bone conduction headphones, and sending detection data and analysis reports to an associated mobile terminal or cloud server via a wireless network. 2.The smart glasses system with microsaccade detection function of claim 1, wherein, It also includes hardware integration modules, specifically including: The smart glasses frame is used to support and fix all functional components, and its temples have built-in wiring channels. The miniaturized near-infrared light source and high-speed miniature camera are integrated in the form of a miniaturized module inside the frame near the eyeball and connected to the main control board through a flexible circuit. The glasses display screen is embedded in front of the lenses in the frame and is used to display visual feedback information; The computing module is an independent wearable or portable processing unit that is connected to the smart glasses frame via a wired or wireless data link. It contains a high-performance processor and storage unit for running the eye movement feature extraction module and the micro-saccade detection and analysis module. 3.The smart glasses system with microsaccade detection function of claim 1, wherein, The eye-tracking image acquisition module includes: The light source driving and synchronization unit is used to drive the near-infrared light-emitting diode to operate in a pulsed manner with a preset duty cycle and generate an electronic trigger signal that is strictly synchronized with each lighting pulse. An optical imaging control unit is used to receive the electronic trigger signal and control the high-speed miniature camera to perform exposure and shooting within the corresponding light source illumination pulse duration, and capture an eye image containing a clear pupil outline and corneal reflective spot; The image buffer and noise suppression unit is used to process the captured raw image data in real time within the camera or in an adjacent buffer memory, perform fixed-pattern noise cancellation and correction of known bad pixels, and acquire raw eye-tracking video data. 4.The smart glasses system with microsaccade detection function of claim 1, wherein, The data processing and transmission module includes: The front-end image optimization unit is used to perform sequential processing on each frame of the original image and apply an adaptive histogram equalization algorithm to enhance the grayscale contrast between the pupil region and the surrounding iris and sclera regions in the image. The optical distortion correction and coordinate system transformation unit is used to digitally correct the shape distortion in the image caused by the lens characteristics according to the pre-calibrated camera lens parameters, restore the true geometric proportions of the scene, and convert the corrected image pixel coordinates into coordinates in a two-dimensional plane coordinate system with the approximate rotation center of the eyeball as the origin, based on the calibration model.

5. The intelligent glasses system with micro-saccade detection function according to claim 1, characterized in that, The data processing and transmission module also includes: The data stream organization and encapsulation unit is used to pack multiple consecutive frames of image data after correction and coordinate transformation, or feature data initially extracted from the image, in chronological order, and add a timestamp and a unique incrementing sequence identifier to each data packet. The adaptive dual-mode link management unit is used to dynamically manage the data transmission path from the glasses to the computing unit. When the wireless signal strength and signal-to-noise ratio are both higher than the preset threshold, data is sent first through the low-latency wireless communication protocol. When the end-to-end latency of wireless transmission is detected to exceed the maximum allowable value or the continuous packet loss rate rises to the warning threshold, it automatically and seamlessly switches to the wired transmission channel composed of physical data interface and connection cable for data transmission. After the wireless link quality recovers to a stable state, it automatically switches back to wireless transmission mode. 6.The smart glasses system with microsaccade detection function of claim 1, wherein, The eye-tracking feature extraction module includes: The pupil positioning and compensation unit is used to analyze the received image sequence frame by frame, locate the center coordinates of the pupil and the coordinates of the corneal reflection point generated by the light source in each frame image, and use the relative positional relationship between the reflection point and the pupil center to calculate and compensate the pupil center coordinates in real time to obtain a pure pupil center coordinate sequence in the eye coordinate system. The trajectory generation and smoothing unit is used to connect the pure pupil center coordinates in chronological order to form the original eye movement trajectory, and to filter the original eye movement trajectory using a digital low-pass filter to output a smooth eye movement trajectory sequence. The kinematic feature calculation unit is used to perform numerical differentiation operations on the smooth eye movement trajectory sequence, calculate the instantaneous velocity and instantaneous acceleration values ​​of the eye movement, and calculate the eye movement direction angle at each sampling moment based on the movement direction of the trajectory points; The feature vector generation and output unit is used to combine and encapsulate the pupil position, instantaneous velocity, instantaneous acceleration and motion direction angle corresponding to each sampling moment to form a multi-dimensional eye movement feature vector arranged in chronological order, and output the eye movement feature vector to the micro-saccade detection and analysis module.

7. The intelligent glasses system with micro-saccade detection function according to claim 5, characterized in that, The adaptive dual-mode link management unit also includes data transmission optimization strategies, specifically: The bandwidth dynamic allocation subunit is used to dynamically adjust the bandwidth allocation of the wireless channel according to the data type being transmitted. When transmitting compressed raw image sequences, high bandwidth is allocated to ensure data integrity. When transmitting only lightweight feature data such as extracted pupil coordinates, bandwidth usage is automatically reduced to save power consumption. The forward error correction and retransmission control subunit is used to apply forward error correction coding to the encapsulated data packets in wireless transmission mode, and perform decoding and error correction at the receiving end. It establishes an acknowledgment and retransmission mechanism based on sequence identifiers, and selectively retransmits only data packets that are not successfully received or fail to be verified, so as to balance the requirements of transmission reliability and real-time performance. The link health assessment subunit is used to continuously monitor the signal strength, signal-to-noise ratio, historical packet loss rate, and latency jitter of the wireless link, and calculate a real-time link health score based on these indicators, which serves as the main decision-making basis for whether to trigger a wired or wireless mode switch.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it enables the operation of the system according to any one of claims 1 to 7.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it enables the operation of the system according to any one of claims 1 to 7.

Images