A mobile terminal-based adaptive visual stimulus presentation and eye movement signal quantification analysis method and system

Abstract

The present application relates to the field of human-computer interaction and computer vision technology, and provides a kind of adaptive visual stimulation presentation and eye movement signal quantification analysis method and system based on mobile terminal.The physical distance between the face of user and screen is calculated in real time, and the rendering parameters of visual target are dynamically adjusted, the vertical synchronization signal of display subsystem is listened to determine the target presentation timestamp, the motion compensation is carried out by fusing inertial measurement unit data, and the eye movement trajectory is reconstructed by combining Kalman filtering processing, the technical problems of non-controlled environment of mobile terminal, such as unstable visual stimulation angle, insufficient time synchronization accuracy and low signal-to-noise ratio of eye movement signal, are solved, high-precision visual stimulation presentation and eye movement characteristic parameter quantification extraction can be realized in non-laboratory environment, and the accuracy, reliability and wide applicability of eye movement evaluation are improved.

Description

Technical Field

[0001] This application relates to the fields of human-computer interaction and computer vision technology, and more specifically, to a method and system for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal. Background Technology

[0002] Eye movement parameters (such as saccade latency, fixation stability, and orientation error rate) are important indicators for assessing human visual attention and executive function. Traditional eye-tracking measurements rely on infrared eye trackers in a laboratory setting, requiring subjects to use a chin rest to stabilize their heads and maintain a constant distance between their eyes and the screen, thereby precisely controlling the angle of visual stimuli.

[0003] However, with the development of mobile healthcare, large-scale, remote eye-tracking assessments using portable devices such as smartphones are becoming a trend. But existing technologies face three core challenges on mobile devices:

[0004] (1) Uncontrollable physical environment: The distance and angle of the user's handheld device change in real time, which causes the projection angle of the stimulus of fixed pixel size on the retina to be inconsistent, thus undermining the standard of psychophysical experiments.

[0005] (2) Low time synchronization accuracy: The multi-task scheduling mechanism of the mobile operating system causes a delay of tens of milliseconds from the issuance of the display command to the actual illumination of the screen, and the camera acquisition frame is not synchronized with the screen refresh frame, which seriously affects the accuracy of latency measurement.

[0006] (3) Low signal-to-noise ratio: The sampling rate of the front camera on the mobile device is usually only 30Hz-60Hz, and it is accompanied by strong hand tremors and light noise, making it difficult to capture fast eye sac events.

[0007] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0008] The purpose of this application is to provide a method and system for adaptive visual stimulus presentation and eye movement signal quantification analysis based on mobile terminals. It has the advantages of being able to achieve high-precision visual stimulus presentation and eye movement feature parameter quantification extraction in non-laboratory environments, thereby improving the accuracy, reliability and wide applicability of eye movement assessment.

[0009] In a first aspect, this application provides a method for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal, wherein the mobile terminal includes a camera, a display subsystem, and an inertial measurement unit, comprising:

[0010] The user's iris region is extracted based on the video stream captured by the camera, and the physical distance between the user's face and the screen is calculated based on the iris pixel diameter.

[0011] Based on the physical distance and preset visual angle parameters, the rendering parameters of the visual target on the screen are determined so that the angle subtended by the visual target to the human eye remains constant.

[0012] Monitor the vertical synchronization signal of the display subsystem and determine the target presentation timestamp based on the vertical synchronization signal;

[0013] The eye position coordinates are extracted from the video stream, motion compensation is performed on the eye position coordinates based on the data collected by the inertial measurement unit, and Kalman filtering is performed on the compensated data to obtain the eye movement trajectory.

[0014] The eye movement trajectory is used to identify saccade events, calculate the latency of the saccade event relative to the target presentation timestamp and the directional deviation angle relative to the visual target, and output the results as a quantitative analysis.

[0015] Furthermore, the physical distance between the user's face and the screen is calculated based on the iris pixel diameter, including:

[0016] The face detection model is used to extract key points of the iris boundary from the video stream, and the pixel diameter of the iris on the image sensor is calculated.

[0017] The physical distance is calculated based on the iris pixel diameter, the preset iris physical diameter, and the equivalent focal length of the camera.

[0018] Furthermore, determining the rendering parameters of the visual target on the screen based on the physical distance and preset visual angle parameters includes:

[0019] The physical size of the visual target is calculated based on the visual angle parameters and the physical distance, and then converted into pixel size according to the screen pixel density.

[0020] Based on the visual angle parameters, the physical distance, and the screen pixel density, calculate the pixel distance between the visual target and the center of the screen;

[0021] Based on the pixel size and the pixel distance, the rendering coordinates and rendering size of the visual target on the screen are determined.

[0022] Furthermore, the monitoring of the vertical synchronization signal of the display subsystem, and the determination of the target presentation timestamp based on the vertical synchronization signal, includes:

[0023] Register a vertical synchronization callback through the frame synchronization interface provided by the mobile terminal operating system, including Android Choreographer or iOS CADisplayLink;

[0024] Upon receiving the vertical synchronization signal, the rendering data of the visual target is written into the frame buffer;

[0025] The system timestamp of the vertical synchronization signal trigger moment is recorded, and the target presentation timestamp is corrected according to the number of frames to be displayed in the frame buffer, the screen refresh cycle, and the physical response delay of the screen panel.

[0026] Furthermore, motion compensation is performed on the eyeball position coordinates based on the data collected by the inertial measurement unit, including:

[0027] The angular velocity data collected by the inertial measurement unit is acquired, and the angular velocity data is integrated within a time window to obtain the head rotation increment matrix;

[0028] The head rotation increment matrix is ​​fused with the head pose matrix obtained by visual detection to obtain the compensated head pose matrix.

[0029] Based on the compensated head posture matrix, the eyeball position coordinates are subjected to inverse rotation transformation to eliminate the displacement component caused by head rotation, thereby obtaining the relative position coordinates of the eyeball in the head coordinate system.

[0030] Furthermore, the step of performing Kalman filtering on the compensated data to obtain the eye-tracking trajectory includes:

[0031] Establish a state vector containing eye position coordinates and eye velocity, and predict the current state vector based on a constant velocity model;

[0032] Using the compensated data as observations, the Kalman gain is calculated, and the predicted state vector and the observations are fused based on the Kalman gain to obtain the filtered state vector.

[0033] The eye position coordinates are extracted from the filtered state vector to form the eye movement trajectory.

[0034] Furthermore, the step of identifying saccade events from the eye movement trajectory includes:

[0035] Set a time window and a dispersion threshold, slide the time window on the eye movement trajectory, and calculate the sum of the maximum span of the gaze point coordinates within the window;

[0036] When the sum of the maximum spans exceeds the dispersion threshold, the time period corresponding to the time window is marked as a saccade state, and the start time of the saccade is recorded;

[0037] When the sum of the maximum spans does not exceed the dispersion threshold, the time period corresponding to the time window is marked as a fixation state.

[0038] Furthermore, the calculation of the latency of the saccade event relative to the target presentation timestamp and the directional deviation angle relative to the visual target includes:

[0039] The difference between the start time of the eye saccade and the timestamp of the target presentation is calculated as the latency period;

[0040] Construct an eye saccade vector and a target vector, calculate the angle between the two vectors as the direction deviation angle, and determine whether the eye saccade direction is correct based on the relationship between the angle and a preset direction determination threshold.

[0041] Furthermore, before extracting the user's iris region from the video stream captured by the camera, the method further includes:

[0042] The system detects whether the physical distance is within a preset effective distance range; when the physical distance is not within the effective distance range, it generates a prompt message to guide the user to adjust the grip position of the mobile terminal.

[0043] Secondly, this application also proposes an adaptive visual stimulus presentation and eye-tracking signal quantification analysis system based on a mobile terminal, wherein the mobile terminal includes a camera, a display subsystem, and an inertial measurement unit, comprising:

[0044] The distance calculation module is used to extract the user's iris region based on the video stream captured by the camera, and to calculate the physical distance between the user's face and the screen based on the iris pixel diameter.

[0045] An adaptive rendering module is used to determine the rendering parameters of the visual target on the screen based on the physical distance and preset visual angle parameters, so that the angle subtended by the visual target to the human eye remains constant.

[0046] The time synchronization module is used to monitor the vertical synchronization signal of the display subsystem and determine the target presentation timestamp based on the vertical synchronization signal.

[0047] The signal processing module is used to extract the eye position coordinates from the video stream, perform motion compensation on the eye position coordinates based on the data collected by the inertial measurement unit, and perform Kalman filtering on the compensated data to obtain the eye movement trajectory.

[0048] The quantitative analysis module is used to identify saccade events from the eye movement trajectory, calculate the latency of the saccade event relative to the target presentation timestamp and the directional deviation angle relative to the visual target, and output the results as quantitative analysis results.

[0049] As can be seen from the above, the adaptive visual stimulus presentation and eye movement signal quantification analysis method and system provided in this application based on mobile terminals solves the technical problems of inconsistent visual stimulus viewing angle, insufficient time synchronization accuracy, and low signal-to-noise ratio of eye movement signals in uncontrolled environments of mobile terminals by calculating the physical distance between the user's face and the screen in real time, dynamically adjusting the rendering parameters of the visual target, listening to the vertical synchronization signal of the display subsystem to determine the target presentation timestamp, fusing inertial measurement unit data for motion compensation, and combining Kalman filtering to reconstruct the eye movement trajectory. It has the advantages of being able to achieve high-precision visual stimulus presentation and eye movement feature parameter quantification extraction in non-laboratory environments, improving the accuracy, reliability, and wide applicability of eye movement assessment. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating the steps of the adaptive visual stimulus presentation and eye-tracking signal quantification analysis method based on a mobile terminal disclosed in an embodiment of the present invention.

[0052] Figure 2 This invention discloses a topological diagram that determines the rendering parameters of a visual target on the screen based on the physical distance and preset visual angle parameters, so that the angle subtended by the visual target to the human eye remains constant.

[0053] Figure 3 This is a schematic diagram of the structure of the adaptive visual stimulus presentation and eye movement signal quantification analysis system based on a mobile terminal disclosed in an embodiment of the present invention. Detailed Implementation

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments belong; the terminology used herein and in the specification of the application is for the purpose of describing particular embodiments only and is not intended to limit these embodiments; the terms "comprising" and "having," and any variations thereof, in the specification of these embodiments and the foregoing drawings, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification of these embodiments and the foregoing drawings are used to distinguish different objects, not to describe a particular order.

[0055] The implementation details of the technical solution in this embodiment are described below:

[0056] This application proposes an adaptive visual stimulus presentation and eye-tracking signal quantification analysis method based on a mobile terminal, wherein the mobile terminal includes a camera, a display subsystem, and an inertial measurement unit, such as... Figure 1 As shown, the method includes:

[0057] S101, extract the user's iris region based on the video stream captured by the camera, and calculate the physical distance between the user's face and the screen based on the iris pixel diameter.

[0058] Furthermore, in S101, calculating the physical distance between the user's face and the screen based on the iris pixel diameter includes: extracting iris boundary key points from the video stream using a face detection model, and calculating the pixel diameter of the iris on the image sensor; and calculating the physical distance based on the iris pixel diameter, a preset iris physical diameter, and the equivalent focal length of the camera.

[0059] Specifically, in this embodiment, the front-facing camera is activated via the app, and the MediaPipe Face Mesh begins capturing 468 key facial points in real time. The system immediately calculates the physical distance D between the phone and the face (e.g., 40cm) based on the iris diameter. If the distance is too close (<30cm) or too far (>50cm), the UI will display a prompt to the user to adjust their grip.

[0060] Achieving high-precision eye tracking on smartphones lacking infrared light sources and corneal reflection hardware requires appearance-based or model-based computer vision algorithms. Currently, Google's MediaPipe FaceMesh is one of the most efficient solutions for mobile devices, capable of inferring 468 3D facial key points in real time from ordinary camera input. The MediaPipe Iris subgraph model precisely locates the center and boundaries of the iris and the eyelid contours. Compared to traditional Haar cascade detection or simple Blob detection, MediaPipe utilizes an attention mesh mechanism, maintaining robustness even when the eyes are partially occluded (e.g., eyelashes, drooping eyelids) or under varying lighting conditions. The resulting iris information not only records eye movements but also determines the distance between the phone screen and the eyes, aiding in the presentation of subsequent stimuli.

[0061] Furthermore, after acquiring the video stream from the mobile terminal's camera, the user's iris region is identified; based on the pixel diameter of the iris and the preset biometric physical diameter, the vertical distance D from the user's face to the screen is calculated in real time.

[0062] Among them, the system pre-stores the average physical diameter d of the human iris. real(Approximately 11.7mm). Images are captured using the front-facing camera, and the iris diameter d is identified using the MediaPipe Iris computer vision model. px Based on the principle of pinhole imaging, the distance D is calculated in real time. The calculation formula is:

[0063]

[0064] Where f is the equivalent focal length of the camera (in pixels).

[0065] S102, based on the physical distance and preset visual angle parameters, determine the rendering parameters of the visual target on the screen so that the angle subtended by the visual target to the human eye remains constant.

[0066] Further, in S102, determining the rendering parameters of the visual target on the screen based on the physical distance and preset visual angle parameters includes: calculating the physical size of the visual target based on the visual angle parameters and the physical distance, and converting it into pixel size based on the screen pixel density; calculating the pixel distance of the visual target from the center of the screen based on the visual angle parameters, the physical distance, and the screen pixel density; and determining the rendering coordinates and rendering size of the visual target on the screen based on the pixel size and the pixel distance.

[0067] Specifically, in this embodiment, the viewing angle is crucial for eye-tracking data, as it establishes the relationship between the eye-tracking data and the experimental stimuli. However, the distance between the user holding the phone and their eyes on a mobile device changes dynamically; therefore, the developed application system must calculate and adjust the pixel positions of the stimuli in real time to ensure a consistent viewing angle.

[0068] A red gaze point appears in the center of the screen, lasting randomly from 1000ms to 2000ms. Before drawing the red gaze point, the algorithm reads the current distance D again and adjusts the position and size of the stimulus accordingly to ensure that the size of the red gaze point on the retina remains constant even if the user experiences slight phone shaking (viewpoint θ = 0.5°).

[0069] To maintain a constant target point eccentricity of θ = 10° (target stimulus) or 0.5° (fixation point) in the anti-saccade task, the system dynamically adjusts the physical size S of the target point in each frame:

[0070]

[0071] In order for the GPU to render this size, physical millimeters need to be converted to pixels (s). px This requires obtaining the screen's pixel density ρ (DPI / 25.4):

[0072]

[0073] Therefore, the comprehensive closed-loop control formula (t represents time) is as follows:

[0074]

[0075] This mechanism eliminates the interference of the user's movement of the phone on the physical properties of the experimental stimuli, ensuring the consistency of measurement conditions.

[0076] See Figure 2 This embodiment describes a topological diagram of how, based on the physical distance and preset visual angle parameters, the rendering parameters of a visual target on the screen are determined to maintain a constant angular subtended by the visual target at the human eye. For example... Figure 2 The mobile terminal captures a video stream of the user's face via its front-facing camera, using the MediaPipe face mesh model to capture 468 key facial points in real time. The system extracts the iris region from the face mesh, identifies the iris boundary, and calculates the iris diameter. Based on the pinhole imaging principle, it calculates the physical distance D between the user's face and the screen according to the iris pixel diameter, a preset physical iris diameter, and the camera's equivalent focal length. For example, the currently calculated distance D is 40 centimeters. The system dynamically adjusts the rendering parameters of the visual stimulus on the screen based on the real-time calculated physical distance D. When a slight shaking of the phone causes a change in distance, the algorithm recalculates the position and size of the stimulus according to the formula that the stimulus size is proportional to D multiplied by tan(θ), where θ is a preset visual angle parameter. This closed-loop control mechanism ensures that the projection size of the visual target on the retina remains constant regardless of how far the user holds the phone, thus maintaining a constant viewing angle θ.

[0077] S103, monitor the vertical synchronization signal of the display subsystem, and determine the target presentation timestamp based on the vertical synchronization signal.

[0078] Further, in S103, the process of monitoring the vertical synchronization signal of the display subsystem and determining the target presentation timestamp based on the vertical synchronization signal includes: registering a vertical synchronization callback through a frame synchronization interface provided by the mobile terminal operating system, wherein the frame synchronization interface includes Android Choreographer or iOS CADisplayLink; writing the rendering data of the visual target into a frame buffer when the vertical synchronization signal is received; recording the system timestamp of the triggering time of the vertical synchronization signal, and correcting the target presentation timestamp based on the number of frames to be displayed in the frame buffer, the screen refresh cycle, and the physical response delay of the screen panel.

[0079] Specifically, in this embodiment, the core measurement metrics for the anti-saccade task include orientation error and saccade latency. Saccade latency is the time difference between stimulus presentation and the onset of eye movement, reflecting cognitive processing load. Because systems like Android are limited by the message queue and layout measurement of the main thread, they cannot guarantee precise frame-level control. If the stimulus presentation time recorded by the application system deviates from the actual screen illumination time, the measured latency data will be meaningless.

[0080] To address rendering latency on Android / iOS systems (60Hz refresh rate), this system does not use conventional high-level UI controls (such as View / UIView). Instead, it directly registers vertical synchronization callbacks through the underlying graphics interface (Android Choreographer or iOS CADisplayLink). The callback trigger time is t. vsync This marks the end of the previous frame and the start of preparation for the new frame.

[0081] Only at the nanosecond level after receiving the V-Sync signal does the system write the stimulus data for the next frame into the frame buffer, awaiting the next screen scan. Therefore, the screen illumination time (t...) ，也是上述的t0）要滞后于回调时间。tphoton的计算公式如下：

[0082]

[0083] 其中，N为图像在缓冲区中排队等待显示的帧数；Tframe是屏幕刷新每一帧所需要的时间；Δthardware为屏幕面板自身的物理响应延迟，通常在30-50ms之间，是一个相对固定的设备常数。

[0084] 基于此，本实施例能够降低刺激呈现的时间精度误差，满足心理物理学测量要求。

[0085] S104，从所述视频流中提取眼球位置坐标，根据所述惯性测量单元采集的数据对所述眼球位置坐标进行运动补偿，并对补偿后的数据进行卡尔曼滤波处理，得到眼动轨迹；

[0086] 进一步，S104中，根据所述惯性测量单元采集的数据对所述眼球位置坐标进行运动补偿，包括：获取所述惯性测量单元采集的角速度数据，对所述角速度数据在时间窗口内积分，得到头部旋转增量矩阵；将所述头部旋转增量矩阵与视觉检测得到的头部姿态矩阵融合，得到补偿后的头部姿态矩阵；根据所述补偿后的头部姿态矩阵，对所述眼球位置坐标进行逆向旋转变换，消除头部转动带来的位移分量，得到眼球在头部坐标系中的相对位置坐标。

[0087] 具体地，本实施例，红色注视点消失后，目标刺激在左侧或右侧出现，呈现1000ms。用户需要抑制看目标的冲动，立刻向相反方向看。摄像头以30Hz的频率持续录制眼球动作。摄像头每33ms获得一帧原始图像数据，同时手机陀螺仪记录下此刻头部 / 手部的微小震动。系统算法从眼球运动向量中减去手部 / 头部的运动向量，防止把手抖、头动误判为眼动。

[0088] 首先，使用IMU消除手抖对头部姿态计算的干扰，再用计算出的精准头部姿态去修正眼动数据。

[0089] 去掉手抖：MediaPipe 算出的头部姿态（Rhead_vis）更新慢（30Hz）且受手抖的影响大；IMU（ωgyro，IMU输出的手机旋转角速度——惯性测量单元采集的角速度数据）更新快可精准反应手部震颤，使用互补滤波将二者融合得到一个稳定的头部姿态，即头部相对于手机的旋转矩阵Rfused。

[0090]

[0091] 其中，Rfused是指经过IMU修正后、干净的头部旋转矩阵；ΔRIMU是由IMU数据ωgyro在Δt时间内积分算出的旋转增量。

[0092] 去掉头动：利用几何变换去除头动。干净头部姿态中的眼动向量Vraw是"眼动+头动”的混合产物，需要将其逆向旋转回头部坐标系。从而获得纯粹的眼球在眼眶中的转动数据（Veye_in_head）。

[0093]

[0094] 进一步，S104中，所述对补偿后的数据进行卡尔曼滤波处理，得到眼动轨迹，包括：建立包含眼球位置坐标和眼球速度的状态向量，根据恒定速度模型预测当前时刻的状态向量；将所述补偿后的数据作为观测值，计算卡尔曼增益，根据所述卡尔曼增益融合预测状态向量和观测值，得到滤波后的状态向量；提取所述滤波后的状态向量中的眼球位置坐标，构成所述眼动轨迹。

[0095] 具体地，本实施例中，智能手机前置摄像头的采样率通常限制在30Hz，且伴随着显著的图像噪声。如何从粗糙、低频的原始信号中提取出精确的眼跳事件，也是本系统开发的技术难点之一。

[0096] 针对处理后的眼动数据（Veye_in_head），采用卡尔曼滤波建立恒定速度模型，假设眼球在极短时间内做匀速运动：

[0097] 状态向量XK：

[0098]

[0099] 包含当前x,y坐标以及x,y方向的速度。

[0100] 预测方程：根据上一时刻的状态（Xk-1|k-1）推测当前时刻的状态（Xk|k-1）：

[0101]

[0102] 其中，F是状态转移矩阵（基于物理公式：位置=旧位置+速度×时间）：

[0103]

[0104] 更新方程：结合实际观测值Zk（MediaPipe输出的坐标）修正预测值：

[0105]

[0106] 其中，Xk|k是指当前时刻的最优估计值；H是观测矩阵，用于将状态变量映射到观测空间的矩阵；KK是卡尔曼增益，决定了更相信预测值还是更相信观测值。KK采用自适应策略，当检测到高速度（眼跳）时，增大过程噪声协方差Q，使KK变大，而从快速跟随眼球的跳动；当速度低时（注视），减小Q，使KK变小，从而过滤掉摄像头的噪点和抖动。

[0107] 基于此，当数据变化慢时（呈现注视点阶段），卡尔曼滤波会降低增益，把曲线磨平；当数据变化大时（呈现目标刺激阶段），它会自适应增大增益，紧跟眼球的快速移动，从而可以输出一条平滑且保留了眼动特征的眼动轨迹曲线。

[0108] S105，从所述眼动轨迹中识别眼跳事件，计算所述眼跳事件相对于所述目标呈现时间戳的潜伏期及相对于所述视觉目标的方向偏差角，输出作为量化分析结果。

[0109] 进一步，S105中，所述从所述眼动轨迹中识别眼跳事件，包括：设定时间窗口和离散度阈值，在所述眼动轨迹上滑动所述时间窗口，计算窗口内注视点坐标的最大跨度之和；当所述最大跨度之和超过所述离散度阈值时，标记该时间窗口对应时段为眼跳状态，并记录眼跳起始时刻；当所述最大跨度之和未超过所述离散度阈值时，标记该时间窗口对应时段为注视状态。

[0110] 具体地，本实施例，眼动仪常使用速度阈值识别（I-VT）算法来分区注视与眼跳，但由于移动设备摄像头的硬件设置，本系统采用离散度阈值算法（I-DT）算法分析系统重构后的轨迹。该方法不依赖于速度，而是依赖位置的分布范围，所以对低频数据的鲁棒性更强。

[0111] 采用离散度阈值算法（I-DT）从重构后的轨迹中分离注视与眼跳事件。该算法的核心逻辑在于眼球在注视时，落点会聚集在一个很小的区域内；而在眼跳时，落点会快速分散。设定该算法每次检查的时间长度为150ms（时间窗Wtime），在当前时间窗内计算X轴与Y轴的最大跨度之和：

[0112]

[0113] 若波动（D）超过1°视角（离散度阈值Dmax）视为眼睛在移动；若D<Dmax，说明眼睛在注视。眼跳开始时刻（tsaccade_start）是上一个注视结束的下一帧。

[0114] 进一步，S105中，所述计算眼跳事件相对于所述目标呈现时间戳的潜伏期及相对于所述视觉目标的方向偏差角，包括：计算所述眼跳起始时刻与所述目标呈现时间戳的差值，作为所述潜伏期，剔除所述潜伏期小于预设最小有效阈值的眼跳事件；构建眼跳向量和目标向量，计算两向量的夹角作为所述方向偏差角，根据所述夹角与预设方向判定阈值的关系，确定眼跳方向是否正确。

[0115] 具体地，本实施例中，计算核心测量指标。潜伏期的计算如下，其中潜伏期小于100ms代表数据无效：

[0116]

[0117] 方向错误的计算：首先构建眼跳向量（眼跳前指向眼跳后）和目标向量（屏幕中心指向刺激目标）。

[0118] 眼跳向量：vsaccade=（xend-xstart，yend-ystart）；目标向量：vtarget=（xtarget-xcenter，ytarget-ycenter）；随后判断这两个向量的方向是否一致：

[0119] 计算两向量的夹角作为方向偏差角θ，采用向量点积公式：

[0120] cos(θ) = (vsaccade · vtarget) / (||vsaccade|| × ||vtarget||)

[0121] θ = arccos(cos(θ))

[0122] 其中，vsaccade · vtarget为两向量的点积，||vsaccade|| 和||vtarget||分别为两向量的模长。

[0123] 根据所述夹角与预设方向判定阈值的关系，确定眼跳方向是否正确：当所述方向偏差角θ小于90度时，判定眼跳方向与目标方向一致，标记为方向错误；当所述方向偏差角θ大于或等于90度时，判定眼跳方向与目标方向相反，标记为方向正确。

[0124] 进一步，在所述根据摄像头采集的视频流提取用户虹膜区域之前，所述方法还包括：检测所述物理距离是否处于预设的有效距离范围内；当所述物理距离未处于所述有效距离范围内时，生成提示信息，引导用户调整移动终端的握持位置。

[0125] 具体地，本实施例中，所述有效距离范围为30厘米至50厘米。当所述物理距离小于30厘米或大于50厘米时，生成提示信息，引导用户调整移动终端的握持位置。该步骤确保用户在合适的距离范围内进行测试，保证后续视觉刺激呈现和眼动信号采集的准确性。

[0126] 第二方面，如图3所示，本申请还提出一种基于移动终端的自适应视觉刺激呈现与眼动信号量化分析系统，所述移动终端包括摄像头、显示子系统以及惯性测量单元，包括：

[0127] 距离计算模块301，用于根据所述摄像头采集的视频流提取用户虹膜区域，并根据虹膜像素直径计算用户面部与屏幕之间的物理距离；

[0128] 自适应渲染模块302，用于根据所述物理距离和预设的视觉角度参数，确定视觉目标在屏幕上的渲染参数，使所述视觉目标对人眼的张角保持恒定；

[0129] 时间同步模块303，用于监听所述显示子系统的垂直同步信号，根据所述垂直同步信号确定目标呈现时间戳；

[0130] 信号处理模块304，用于从所述视频流中提取眼球位置坐标，根据所述惯性测量单元采集的数据对所述眼球位置坐标进行运动补偿，并对补偿后的数据进行卡尔曼滤波处理，得到眼动轨迹；

[0131] 量化分析模块305，用于从所述眼动轨迹中识别眼跳事件，计算所述眼跳事件相对于所述目标呈现时间戳的潜伏期及相对于所述视觉目标的方向偏差角，输出作为量化分析结果。

[0132] 以用于执行第一方面所述的基于移动终端的自适应视觉刺激呈现与眼动信号量化分析方法，在此不再赘述。

[0133] 以上所述仅为本申请的实施例而已，并不用于限制本申请的保护范围，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims

1. A method for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal, wherein the mobile terminal includes a camera, a display subsystem, and an inertial measurement unit, characterized in that, include: The user's iris region is extracted based on the video stream captured by the camera, and the physical distance between the user's face and the screen is calculated based on the iris pixel diameter. Based on the physical distance and preset visual angle parameters, the rendering parameters of the visual target on the screen are determined so that the angle subtended by the visual target to the human eye remains constant. Monitor the vertical synchronization signal of the display subsystem and determine the target presentation timestamp based on the vertical synchronization signal; The eye position coordinates are extracted from the video stream, motion compensation is performed on the eye position coordinates based on the data collected by the inertial measurement unit, and Kalman filtering is performed on the compensated data to obtain the eye movement trajectory. The eye movement trajectory is used to identify saccade events, calculate the latency of the saccade event relative to the target presentation timestamp and the directional deviation angle relative to the visual target, and output the results as a quantitative analysis.

2. The adaptive visual stimulus presentation and eye-tracking signal quantification analysis method based on a mobile terminal according to claim 1, characterized in that, The physical distance between the user's face and the screen is calculated based on the diameter of the iris pixels, including: The face detection model is used to extract key points of the iris boundary from the video stream, and the pixel diameter of the iris on the image sensor is calculated. The physical distance is calculated based on the iris pixel diameter, the preset iris physical diameter, and the equivalent focal length of the camera.

3. The adaptive visual stimulus presentation and eye-tracking signal quantification analysis method based on a mobile terminal according to claim 2, characterized in that, The process of determining the rendering parameters of the visual target on the screen based on the physical distance and preset visual angle parameters includes: The physical size of the visual target is calculated based on the visual angle parameters and the physical distance, and then converted into pixel size according to the screen pixel density. Based on the visual angle parameters, the physical distance, and the screen pixel density, calculate the pixel distance between the visual target and the center of the screen; Based on the pixel size and the pixel distance, the rendering coordinates and rendering size of the visual target on the screen are determined.

4. The adaptive visual stimulus presentation and eye-tracking signal quantification analysis method based on a mobile terminal according to claim 3, characterized in that, The monitoring of the vertical synchronization signal of the display subsystem, and the determination of the target presentation timestamp based on the vertical synchronization signal, include: Register a vertical synchronization callback through the frame synchronization interface provided by the mobile terminal operating system, including Android Choreographer or iOS CADisplayLink; Upon receiving the vertical synchronization signal, the rendering data of the visual target is written into the frame buffer; The system timestamp of the vertical synchronization signal trigger moment is recorded, and the target presentation timestamp is corrected according to the number of frames to be displayed in the frame buffer, the screen refresh cycle, and the physical response delay of the screen panel.

5. The method for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal according to claim 4, characterized in that, Motion compensation is performed on the eyeball position coordinates based on the data collected by the inertial measurement unit, including: The angular velocity data collected by the inertial measurement unit is acquired, and the angular velocity data is integrated within a time window to obtain the head rotation increment matrix; The head rotation increment matrix is ​​fused with the head pose matrix obtained by visual detection to obtain the compensated head pose matrix. Based on the compensated head posture matrix, the eyeball position coordinates are subjected to inverse rotation transformation to eliminate the displacement component caused by head rotation, thereby obtaining the relative position coordinates of the eyeball in the head coordinate system.

6. The adaptive visual stimulus presentation and eye-tracking signal quantification analysis method based on a mobile terminal according to claim 5, characterized in that, The process of performing Kalman filtering on the compensated data to obtain the eye movement trajectory includes: Establish a state vector containing eye position coordinates and eye velocity, and predict the current state vector based on a constant velocity model; Using the compensated data as observations, the Kalman gain is calculated, and the predicted state vector and the observations are fused based on the Kalman gain to obtain the filtered state vector. The eye position coordinates are extracted from the filtered state vector to form the eye movement trajectory.

7. The method for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal according to claim 6, characterized in that, The process of identifying saccade events from the eye movement trajectory includes: Set a time window and a dispersion threshold, slide the time window on the eye movement trajectory, and calculate the sum of the maximum span of the gaze point coordinates within the window; When the sum of the maximum spans exceeds the dispersion threshold, the time period corresponding to the time window is marked as a saccade state, and the start time of the saccade is recorded; When the sum of the maximum spans does not exceed the dispersion threshold, the time period corresponding to the time window is marked as a fixation state.

8. The method for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal according to claim 7, characterized in that, The calculation of the latency of the saccade event relative to the target presentation timestamp and the directional deviation angle relative to the visual target includes: The difference between the start time of the eye saccade and the timestamp of the target presentation is calculated as the latency period; Construct an eye saccade vector and a target vector, calculate the angle between the two vectors as the direction deviation angle, and determine whether the eye saccade direction is correct based on the relationship between the angle and a preset direction determination threshold.

9. The method for adaptive visual stimulus presentation and eye-tracking signal quantification analysis based on a mobile terminal according to claim 1, characterized in that, Before extracting the user's iris region from the video stream captured by the camera, the method further includes: The system detects whether the physical distance is within a preset effective distance range; when the physical distance is not within the effective distance range, it generates a prompt message to guide the user to adjust the grip position of the mobile terminal.

10. A mobile terminal-based adaptive visual stimulus presentation and eye-tracking signal quantification analysis system, wherein the mobile terminal includes a camera, a display subsystem, and an inertial measurement unit, characterized in that... include: The distance calculation module is used to extract the user's iris region based on the video stream captured by the camera, and to calculate the physical distance between the user's face and the screen based on the iris pixel diameter. An adaptive rendering module is used to determine the rendering parameters of the visual target on the screen based on the physical distance and preset visual angle parameters, so that the angle subtended by the visual target to the human eye remains constant. The time synchronization module is used to monitor the vertical synchronization signal of the display subsystem and determine the target presentation timestamp based on the vertical synchronization signal. The signal processing module is used to extract the eye position coordinates from the video stream, perform motion compensation on the eye position coordinates based on the data collected by the inertial measurement unit, and perform Kalman filtering on the compensated data to obtain the eye movement trajectory. The quantitative analysis module is used to identify saccade events from the eye movement trajectory, calculate the latency of the saccade event relative to the target presentation timestamp and the directional deviation angle relative to the visual target, and output the results as quantitative analysis results.

Images