Method, device and equipment for extracting multi-dimensional eye movement features

Abstract

The present invention provides a method, device and equipment for extracting multi-dimensional eye movement features. Relating to the field of data processing technology, the method includes: when a subject performs an eye movement test according to a predetermined eye movement task, collecting the subject's viewpoint position information on the test screen according to a preset sampling frequency and recording the sampling time corresponding to each viewpoint position information; determining the data of the saccade latency stage, execution stage and stabilization stage according to each viewpoint position information and sampling time; determining the time-domain static features of the eye movement according to the data of the saccade latency stage, execution stage and stabilization stage; and determining the time-domain dynamic features of the eye movement according to the data of the saccade latency stage; generating health prompt information according to the time-domain static features and the time-domain dynamic features. The present invention can extract accurate, effective and comprehensive multi-dimensional eye movement features from eye movement data at multiple time scales, so that the health prompt information generated for the subject is more accurate and comprehensive.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and equipment for extracting multi-dimensional eye movement features. Background Art

[0002] Eye movement testing is a method that tracks physiological parameters such as the subject's eye movement trajectory, fixation point, and speed as they move across a test screen. Because eye movement is coordinated and controlled by multiple brain regions (including the frontal oculomotor area, parietal lobe, basal ganglia, thalamus, and brainstem), the results of eye movement testing can reflect specific brain functions, such as attention allocation, visual information processing, emotional state, memory, inhibitory control, and other cognitive and physiological conditions. Therefore, eye movement data obtained from eye movement testing can reveal potential health risks in a subject's cognitive, emotional, and neurological functions.

[0003] In related technologies, the method for acquiring and processing eye movement data typically involves presenting a fixation point on a test screen and asking the subject to look toward it. Then, a number of stimulus points appear, and the subject is asked to look from the fixation point to the stimulus point. An eye tracker is used to track the subject's eye position, movement trajectory, and other eye movement data during the test. The eye tracker pre-stores a judgment threshold, and the collected eye movement data is compared with the corresponding judgment threshold. Based on the comparison results, the subject is predicted to have health risks and health warning information is sent to the subject.

[0004] However, traditional methods can only rely on eye trackers to obtain single-dimensional eye movement data. During the actual eye movement test, the subjects are easily disturbed by various factors such as ambient lighting and eye health, resulting in inaccurate eye movement data collected. Therefore, single-dimensional eye movement data cannot accurately reflect the subjects' true cognitive changes and physiological status, making it difficult to achieve accurate predictions. The error rate is high, resulting in inaccurate health warning information sent. Summary of the Invention

[0005] The embodiments of the present invention provide a method, device and equipment for extracting multi-dimensional eye movement features to solve the problem in the prior art that single-dimensional eye movement data cannot accurately reflect the real cognitive changes and physiological state of the subject, making it difficult to achieve accurate prediction and having a high error rate, thereby leading to inaccurate health warning information sent.

[0006] In a first aspect, an embodiment of the present invention provides a method for extracting multi-dimensional eye movement features, comprising:

[0007] When the subject performs an eye movement test according to a predetermined saccade task, the subject's viewpoint position information on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded;

[0008] Determining data of a saccade latency phase, a saccade execution phase, and a saccade stabilization phase according to each viewpoint position information and a sampling time corresponding to each viewpoint position information;

[0009] Determining temporal static features of eye movements based on the data of the saccade latency stage, the data of the saccade execution stage, and the data of the saccade stabilization stage; and determining temporal dynamic features of eye movements based on the data of the saccade latency stage;

[0010] Health reminder information is generated based on the time domain static features and the time domain dynamic features.

[0011] In the second aspect, an embodiment of the present invention provides a device for extracting multi-dimensional eye movement features, which is characterized in that it includes: a data acquisition module for collecting the subject's viewpoint position information on the test screen according to a preset sampling frequency and recording the sampling time corresponding to each viewpoint position information when the subject performs an eye movement test according to a predetermined eye movement task; a data processing module for determining the data of the eye movement latency stage, the eye movement execution stage and the eye movement stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; a feature extraction module for determining the time domain static features of eye movement according to the data of the eye movement latency stage, the eye movement execution stage and the eye movement stabilization stage; and determining the time domain dynamic features of eye movement according to the data of the eye movement latency stage; an information generation module for generating health prompt information according to the time domain static features and the time domain dynamic features.

[0012] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method in the first aspect or any possible implementation of the first aspect is implemented.

[0013] In an embodiment of the present invention, when a subject performs an eye movement test according to a predetermined eye movement task, the viewpoint position information of the subject on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded; according to each viewpoint position information and the sampling time corresponding to each viewpoint position information, the data of the eye movement latency stage, the data of the eye movement execution stage and the eye movement stabilization stage are determined; according to the data of the eye movement latency stage, the data of the eye movement execution stage and the eye movement stabilization stage, the time domain static characteristics of the eye movement are determined; and, according to the data of the eye movement latency stage, the time domain dynamic characteristics of the eye movement are determined; and according to the data of the eye movement latency stage, the health prompt information is generated according to the time domain static characteristics and the time domain dynamic characteristics. The embodiment of the present invention divides the entire eye movement process into multiple scales from the time dimension, and determines the eye movement data of the three stages: the saccade latency stage, the execution stage, and the stabilization stage. This division method is closer to the physiological mechanism of eye movement. Then, the time domain static features of the eye movement are extracted by combining the data of the three stages, and the dynamic features are extracted for the latent stage. The effectiveness, accuracy, and comprehensiveness of the eye movement features are improved based on the multi-dimensional eye movement features of time domain static and dynamic, so that a more comprehensive health assessment of the subjects can be performed, and the health prompt information finally generated is also more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is an application scenario diagram of the method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention;

[0015] Figure 2 is a flowchart of an implementation method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention;

[0016] Figure 3 This is a schematic diagram of test screen coordinate conversion provided by an embodiment of the present invention;

[0017] Figure 4 Schematic diagram of a multi-dimensional feature extraction architecture provided by an embodiment of the present invention;

[0018] Figure 5 2 is a schematic diagram of a forward saccade task design provided by an embodiment of the present invention;

[0019] Figure 6 2 is a schematic diagram of a reverse saccade task design provided by an embodiment of the present invention;

[0020] Figure 7 Schematic diagram of the design of the memory saccade task provided by an embodiment of the present invention;

[0021] Figure 8 This is an application scenario diagram of a method for extracting multi-dimensional eye movement features provided by another embodiment of the present invention;

[0022] Figure 92 is a schematic structural diagram of a device for extracting multi-dimensional eye movement features provided by an embodiment of the present invention;

[0023] Figure 10 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] Based on the defects of the existing technology, this application proposes the following technical ideas: divide the entire eye movement process into multiple scales from the time dimension, and determine the eye movement data of the three stages: the saccade latency stage, the execution stage, and the stabilization stage. Since the control areas of the brain corresponding to different stages are different, this division method is closer to the physiological mechanism of eye movement. Then, the time domain static features of the eye movement are extracted by combining the data of the three stages, and the dynamic features are extracted for the latent stage. According to the multi-dimensional eye movement features such as time domain static and dynamic, it is judged whether the subject has potential health risks. Through stage division and multi-dimensional feature extraction, the effectiveness, accuracy and comprehensiveness of the features can be improved, a more comprehensive health assessment of the subject can be carried out, and the health prompt information finally generated is more accurate.

[0026] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] Figure 1 A diagram illustrating an application scenario of the method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention.

[0028] like Figure 1 As shown, the application scenario provided by this embodiment includes: an eye tracker 11 and a test screen 12; wherein, the test screen 12 can be, but is not limited to, a screen of a display that communicates with a server.

[0029] The subject wears an eye tracker 11 and quickly looks at the fixation point (FP) at the center of the test screen 12 at the beginning of the eye movement test. Then, when the fixation point FP disappears, a stimulus point appears on the test screen. The subject needs to quickly look at the stimulus point as required. During this process, the eye tracker collects the subject's eye image according to a certain sampling frequency domain, and then processes the eye image to obtain the subject's viewpoint position on the test screen 12. The viewpoint position sequence corresponding to all sampling moments constitutes an eye movement trajectory of the eye movement test process.

[0030] It is understandable that the application scenario provided in this embodiment is only used as an example to facilitate relevant personnel to understand the method provided by the present invention, and does not constitute any limitation to the present invention.

[0031] The method provided by the present invention will be described in detail below in conjunction with application scenarios.

[0032] See also Figure 2 , which shows a flowchart of the implementation of the method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention, and the detailed steps are as follows:

[0033] S201 , when a subject performs an eye movement test according to a predetermined saccade task, the viewpoint position information of the subject on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded.

[0034] In this step, saccades can be divided into reflexive saccades and autonomous saccades according to the driving factors of saccades, and the corresponding judgment test task paradigm can be designed according to the driving factors of saccades.

[0035] In one possible implementation, a progressive saccade task (PS) is designed to address subjects' reflexive saccades. At the beginning of each trial, a fixation point (FP) is displayed in the center of the test screen. Subjects are instructed to quickly look toward the FP and maintain fixation until a time requirement is met. Then, a stimulus point randomly appears outside the FP, and subjects are instructed to quickly look toward the stimulus point.

[0036] In one possible implementation, an anti-saccade task (AS) is designed to test the subject's autonomous eye movements. In the anti-saccade task AS, the subject needs to look at the fixation point FP as quickly as possible and maintain fixation until the time requirement is met. Then, a stimulus point appears randomly outside the fixation point FP, and the subject needs to quickly look at the target position opposite to the stimulus point (that is, the position of the symmetrical point of the stimulus point relative to the fixation point).

[0037] In one possible implementation, a memory-guided saccade task (MGS) is designed to test the subject's autonomous saccades. At the beginning of the trial, the test screen displays the fixation point FP, and then stimulus points are randomly displayed outside the fixation point. The stimulus points then disappear. At this time, the subject still keeps looking at the fixation point FP. During the delay period, the subject needs to remember the position of the stimulus point and quickly look at the target position where the stimulus point appeared when the fixation point FP disappears.

[0038] It should be noted that the detailed processes of the above three saccade tasks will be further described in the following embodiments.

[0039] In this step, the sampling frequency is preset to be at least 1000 times / second. The sampling device can be, but is not limited to, an infrared video-based eye tracker. The viewpoint position information is the visual angle coordinates of the viewpoint relative to the center of the test screen.

[0040] In one possible implementation, when a subject performs an eye movement test according to a predetermined saccade task, an infrared video-based eye tracker illuminates the subject's eyes with infrared light, and a built-in camera of the eye tracker records eye images. From the beginning to the end of a saccade task, a set of eye images is collected. Then, each frame of the eye image set is processed using a computer vision algorithm to obtain the subject's eye features, including pupil center coordinates, pupil size, and corneal reflection point coordinates. Based on the above eye features and the relative positional relationship between the eye and the test screen, the viewpoint coordinates are calculated. The viewpoint coordinates are pixel coordinates. The pixel coordinates of each viewpoint are then converted into the visual angle coordinates of the viewpoint relative to the center of the test screen, that is, the viewpoint position information is obtained, and the sampling time corresponding to each viewpoint position information is recorded to obtain a position-time series.

[0041] Exemplary, reference Figure 3 , the resolution of the test screen is 1920*1080 pixels. The coordinate system of the test screen is established with the upper left corner of the test screen as the origin. The pixel coordinates of the upper left corner are (0, 0), the pixel coordinates of the lower right corner are (1920, 1080), the pixel coordinates of the center point of the screen are (960, 540), and the pixel coordinates of the viewpoint are marked as (eyeX, eyeY). The viewing angle coordinates corresponding to the viewpoint are ( , ), pdd represents the number of pixels corresponding to each viewing angle; the value of pdd needs to be determined based on the distance between the subject's eyes and the screen when the subject is actually performing an eye movement test.

[0042] In this step, the sampling frequency is set to at least 1000 times / second. The high sampling frequency can capture subtle changes in the eyes and help extract high-dimensional eye movement features. In addition, in eye movement research, converting pixel coordinates into visual angle coordinates and using viewing angles can more accurately describe eye movement and gaze position, which is not affected by screen resolution or display device size and can more intuitively reflect the eye's attention to information at different positions.

[0043] S202 : determining data of a saccade latency stage, data of a saccade execution stage, and data of a saccade stabilization stage according to each viewpoint position information and a sampling time corresponding to each viewpoint position information.

[0044] In this step, the saccade latency phase is the period between the disappearance of the fixation point and the onset of the main saccade. This phase reflects the brain's processing of visual information, decision-making, and generation of motor commands, reflecting the neural efficiency of the brain's transformation from perception to movement, and is considered the preparation phase. The saccade execution phase is the phase from the onset of the main saccade to the rapid eye movement to the target location. This phase, considered the rapid saccade phase, reflects the brain's control of eye movement. The saccade stabilization phase is the phase after the last saccade is completed, during which the eyes maintain a stable fixation on the target location. During this phase, the brain needs to maintain sustained attention to the target location and suppress reflexive saccades triggered by irrelevant stimuli.

[0045] According to the position-time sequence obtained in step S201 (ie, each viewpoint position and the sampling time corresponding to the viewpoint position), data of the three stages are determined respectively according to the three divided stages.

[0046] In a possible implementation, the data of the saccade latency stage includes: saccade latency; the data of the saccade execution stage includes: saccade amplitude, saccade duration and saccade rate; the data of the saccade stabilization stage includes: saccade landing point error;

[0047] It should be noted that the detailed calculation method of the data at each stage will be described in detail in the following embodiments.

[0048] S203, determining the temporal static features of the eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and determining the temporal dynamic features of the eye movement according to the data of the saccade latency stage.

[0049] In this step, in addition to determining the time domain static features based on the data of the three stages, time domain dynamic features are also extracted for the data of the incubation period. In health monitoring, dynamic features can capture subtle anomalies from a higher dimension that traditional static features cannot reflect.

[0050] In one possible implementation, the time domain static features may include: average saccade latency, average saccade amplitude, average saccade duration, average saccade landing error, and saccade peak rate. Since a single test data may have occasional anomalies caused by environmental factors, in this embodiment, the average is taken for each data. Through these static features, it is possible to comprehensively judge whether the subject has potential health risks. For example, an extended latency may indicate degeneration of the prefrontal cortex and basal ganglia function, leading to delayed saccade initiation. These static features can capture early abnormalities in nervous system function by quantifying the initiation efficiency, movement trajectory, accuracy, and speed of saccades.

[0051] Because the duration and stability of the latency phase serve as important indicators for studying cognitive function and neurological disorders, extracting high-dimensional features of the latency phase can help reveal early cognitive decline in subjects. Stability can be measured by the standard deviation of eye acceleration during the latency phase. Therefore, in one possible implementation, temporal dynamic features may include the standard deviation of eye acceleration. Eye acceleration reflects the rate of change of eye movement velocity during the latency phase. The standard deviation quantifies the temporal dispersion of eye acceleration and can reflect the stability of the subject during the latency phase. In health monitoring, the standard deviation of eye acceleration during the saccade latency phase essentially reflects the dynamic stability of the neuromuscular system during saccade initiation. An increase in the standard deviation of eye acceleration during the saccade latency phase may indicate central control disturbances, increased motor planning noise, or underlying pathology. It can serve as an auxiliary indicator for assessing neurological diseases in clinical health monitoring. For normal subjects, attention is continuously maintained on the fixation point during the latency phase, and theoretically, rapid eye movements are not expected. Therefore, eye acceleration in healthy subjects is very low. If the standard deviation of the subject's eye acceleration during the latent stage is large, it means that the subject's eye acceleration fluctuates greatly during the latent stage, and there may be potential problems in the neural control mechanism during the movement preparation stage. For example, the subject is anxious and has difficulty maintaining concentration, or abnormal regulation of the basal ganglia of the brain causes the subject to be unable to accurately control eye movements.

[0052] S204: Generate health reminder information based on the time-domain static features and the time-domain dynamic features.

[0053] In this step, after obtaining the time domain static features and time domain dynamic features, each feature can be compared with the corresponding judgment threshold, and the comparison results of each feature can be combined to generate targeted health prompt information for the subject, which can more accurately and comprehensively evaluate the subject's nervous system function.

[0054] For example, the normal range of standard deviation of latency acceleration in healthy adults is , the judgment threshold of this feature can be set to If the standard deviation of the subject's eye acceleration is greater than the threshold and the peak rate of saccades is lower than the threshold, a health prompt message indicating damage to the basal ganglia-cortex circuit is generated.

[0055] In this embodiment, the entire eye movement process is divided into multiple scales from the time dimension, and the eye movement data of the three stages, namely the saccade latency stage, the execution stage and the stabilization stage, are determined respectively. This division method is closer to the physiological mechanism of eye movement. Then, the time domain static features of the eye movement are extracted by combining the data of the three stages, and the dynamic features are extracted for the latent stage. The multi-dimensional eye movement features such as time domain static and dynamic are used to judge whether the subject has potential health risks. The static features can capture subtle abnormal changes in nervous system function before the clinical symptoms of neurodegenerative diseases appear by quantifying the initiation efficiency, movement trajectory, accuracy and speed of saccades. The dynamic features make up for the defects of the static features and capture subtle abnormalities that the static features cannot reflect from a higher dimension. The combination of static and dynamic multi-dimensional features can conduct a more comprehensive health assessment of the subject, and the health prompt information finally generated is more accurate.

[0056] In a possible embodiment, the data of the saccade latency stage include: saccade latency; the data of the saccade execution stage include: saccade amplitude, saccade duration and saccade rate; the data of the saccade stabilization stage include: saccade landing point error. The data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage are determined according to each viewpoint position information and the sampling time corresponding to each viewpoint position information, including: determining the start time and end time of the main saccade in each trial saccade task, as well as the start time and end time of each corrective saccade according to the viewpoint position information and the sampling time corresponding to each viewpoint position information; wherein the main saccade is the first saccade that occurs during the subject's scanning process from the fixation point to the target position, and the corrective saccade is the saccade that occurs after the end time of the main saccade; the absolute value of the difference between the time when the fixation point disappears on the test screen in each trial saccade task and the start time of the main saccade is used as the saccade of each trial saccade task. incubation period; determining the saccade amplitude and saccade rate of each corrective saccade according to the viewpoint position information corresponding to the start time of each corrective saccade and the viewpoint position information corresponding to the end time of each corrective saccade; taking the absolute value of the difference between the start time of each corrective saccade and the end time of each corrective saccade as the saccade duration of each corrective saccade; determining the saccade landing point error of each trial saccade task according to the viewpoint position information corresponding to the target position on the test screen and the end time of the last corrective saccade in each trial saccade task; wherein the fixation point is the center point of the test screen, and the target position is the position of the stimulus point displayed on the test screen or the position of the symmetrical point of the stimulus point relative to the center point.

[0057] It should be noted that conventional methods are all for primary saccades for the research of saccade related parameters mostly, that is, after the disappearance of the fixation point and the occurrence of the stimulus point, the first saccade that occurs from the fixation point to the target position scanning process. Under normal circumstances, saccades are eye movements that are fast and accurate, which help us to transfer our sight to the target position. If the first saccade does not accurately arrive at the target position, corrective saccades will be produced. But the experimenter, when carrying out the saccade task test, is easy to be disturbed by various factors such as environmental factors and human factors, causes primary saccades to be abnormal, and multiple corrective saccades can occur afterwards. In this case, analyzing the related parameters of primary saccades separately is difficult to fully reflect the true state of the experimenter. Therefore, the present embodiment, except studying the related parameters of primary saccades, also focuses on the related parameters of corrective saccades, with comprehensive and accurate analysis of the true state of the experimenter.

[0058] It should be noted that in this embodiment, the fixation point is the center point of the test screen, and the target position is the position of the stimulus point displayed on the test screen or the position of a point symmetrical to the stimulus point relative to the center point. Specifically, when the saccade task is a forward saccade task, the target position is the position of the stimulus point; when the saccade task is a backward saccade task, the target position is the position symmetrical to the fixation point; when the saccade task is a memory saccade task, the target position is the position where the stimulus point appeared.

[0059] In actual saccade task testing scenarios, after the fixation point disappears, the subject's primary saccade landing point may have a significant error relative to the target location. Subsequently, multiple corrective saccades may occur over a period of time to reduce the error. Large errors in primary saccade landing points may be due to health concerns, head rotation, or interference from external environmental factors. Healthy subjects will quickly adjust their gaze toward the target location when the primary saccade landing point error is large, and corrective saccades will occur during this viewpoint correction process. Subjects with potential early neurological dysfunction will also adjust their viewpoint when the primary saccade landing point error is large, resulting in corrective saccades. However, the speed and duration of these adjustments, as well as the resulting viewpoint error, will differ from those of healthy subjects. Abnormal corrective saccades are essentially the brain's compensatory response to eye movement errors, and changes in their frequency or pattern can serve as a marker of neurological dysfunction. For example, abnormalities in motor initiation and regulation in the basal ganglia of neurological disorders such as Parkinson's and Huntington's can lead to delayed saccade initiation and abnormal amplitude, resulting in an increase in corrective saccades and unstable saccade amplitude. Another example is Alzheimer's disease, where the cerebral cortex (especially the prefrontal and parietal lobes) atrophies, leading to damage to the eye movement control network. During saccade tasks, the number of corrective saccades increases, and the duration of these saccades increases while the peak rate decreases, reflecting a decline in target location and motor planning abilities. Therefore, in this embodiment, by extracting the features of corrective saccades (the amplitude, rate, and duration of corrective saccades), it can assist in monitoring early abnormalities in nervous system function and provide an important basis for revealing abnormal brain function.

[0060] In a possible implementation, the eye saccade detection method may adopt the following steps:

[0061] Step 1: According to the viewpoint position information and the sampling time corresponding to each viewpoint position information, calculate the instantaneous rate corresponding to each sampling time and obtain the rate-time series (denoted as ).in, represents the sampling time, and i represents the number of samples.

[0062] In one possible implementation, the instantaneous rate of the viewpoint position corresponding to each sampling time is calculated based on the adjacent difference method. Specifically, the viewpoint position-time series is expressed as ( ), represents the horizontal viewing angle of the viewpoint corresponding to the i-th sampling time, represents the vertical viewing angle of the viewpoint corresponding to the i-th sampling time. Then the instantaneous speed of each viewpoint in the horizontal direction is: , the instantaneous velocity in the vertical direction is: , Represents the sampling time interval. The instantaneous rate at the viewpoint position corresponding to each sampling time is: .

[0063] Step 2: Set a speed threshold, which can be in the range of 15° / s to 30° / s. Consecutive viewpoints that are longer than the speed threshold for more than 20 milliseconds are determined as saccade segments, with the first point being the starting point of the saccade and the last point being the ending point of the saccade.

[0064] Step 3: Calculate the amplitude between the start and end points of each saccade segment.

[0065] Step 4: The first saccade segment in the process of scanning towards the target position after the fixation point disappears is defined as the main saccade, and the saccade segment with an amplitude greater than 2 visual angles after the end of the main saccade is defined as the corrective saccade.

[0066] In one possible implementation, assuming that the time when the fixation point disappears on the test screen in each saccade task is t1 and the start time of the main saccade is t2, the saccade latency of each saccade task is .

[0067] In a possible implementation, the start time of each corrective saccade is t3, and the corresponding viewpoint position information is recorded as ( The end time of each corrective saccade is t4, and the corresponding viewpoint position information is recorded as ( ), then the duration of each corrective saccade is T= ; Saccade amplitude of each corrective saccade ,in, , ; Saccadic rate of each corrective saccade ,in, , .

[0068] In one possible implementation, the target position on the test screen in each saccade task is recorded as ( ), the viewpoint position information corresponding to the end time of the last corrective saccade is recorded as ( ), then the saccade landing error of each trial saccade task is .

[0069] In a possible embodiment, determining the temporal static features of the eye movement based on the data of the saccade latency stage, the data of the saccade execution stage, and the data of the saccade stabilization stage includes:

[0070] According to the saccadic latency of each trial saccadic task, the average saccadic latency corresponding to the n-trial saccadic task is calculated; wherein n is an integer greater than zero. The average saccadic latency corresponding to the n-trial saccadic task .

[0071] According to the eye saccade landing error of each trial eye saccade task, the average eye saccade landing error corresponding to the n-trial eye saccade task is calculated. .

[0072] According to the saccade amplitude of each corrective saccade, the average saccade amplitude of the corrective saccade in each trial saccade task is calculated; the average saccade amplitude , where m is the number of corrective saccades in each trial.

[0073] According to the saccade duration of each corrective saccade, the average saccade duration of the corrective saccade in each trial saccade task is calculated; the average saccade duration .

[0074] According to the saccade rate of each corrective saccade, the peak saccade rate of the corrective saccade in each trial saccade task is calculated; the peak saccade rate is the maximum rate among multiple corrective saccades.

[0075] In this embodiment, the core significance of average saccade latency is the time interval from the onset of visual stimulation to saccade initiation, reflecting the efficiency of neural signal transmission, decision initiation, and motor planning. It can be used to monitor early health risks related to neurodegenerative diseases. For example, a prolonged latency may indicate degeneration of the prefrontal cortex and basal ganglia, leading to delayed saccade initiation (research shows that the latency of early-stage Alzheimer's patients is 15%-20% longer than that of healthy controls; in Parkinson's disease patients, due to dopamine neuron damage, the efficiency of neural signal integration during the saccade preparation phase is reduced, and the latency can be extended by 10%-15%). The core significance of mean saccade amplitude is the spatial displacement of saccades, reflecting the ability to locate visual targets and the coordination of the oculomotor system. It can indicate early-stage motor control disorders in the nervous system. Mean saccade duration, which is correlated with saccade rate, reflects the dynamic characteristics of eye movements. In the early stages of neurodegenerative diseases, saccade duration is prolonged (for example, saccade duration in Alzheimer's disease is 10%-15% longer than in healthy individuals), suggesting attenuation of extraocular motor control signals, possibly related to degeneration of basal ganglia-cerebellar circuit function. Mean saccade landing error reflects the ability to regulate visual feedback and correct movement. Large landing error indicates abnormalities in the brain's regulatory system. Saccade peak velocity reflects the explosiveness of eye movements and depends on the contractility of extraocular muscles and the strength of neural drive. For example, in patients with early Parkinson's disease, peak velocity is reduced due to dopamine deficiency.

[0076] In this embodiment, by dividing primary saccades into primary saccades and corrective saccades and focusing on the characteristics of corrective saccades, compared with the traditional method of only studying the characteristics of primary saccades, more subtle functional abnormalities in the early stages of nervous system dysfunction can be captured, making up for the defect that the characteristics of primary saccades cannot fully reflect the true condition of the subjects, and improving the accuracy of early health problem monitoring.

[0077] In a possible embodiment, the data of the saccadic latency stage further includes: eye acceleration corresponding to each sampling moment in the saccadic latency stage; determining the data of the saccadic latency stage based on each viewpoint position information and the sampling time corresponding to each viewpoint position information, further includes:

[0078] Calculating the instantaneous rate corresponding to each sampling time according to each viewpoint position information and the sampling time corresponding to each viewpoint position information;

[0079] The eye acceleration corresponding to each sampling time in the saccadic latency stage is determined according to each sampling time and the instantaneous rate corresponding to each sampling time.

[0080] It should be noted that the calculation method of the instantaneous rate corresponding to each sampling time can refer to the introduction in the above embodiment. The instantaneous rate at the viewpoint position corresponding to each sampling time is: Then the horizontal component of acceleration at each viewpoint is , the vertical component of acceleration is: , and finally the total acceleration at each viewpoint is According to the time of the latent stage in each trial of the saccade task, the acceleration sequence of the latent stage is extracted.

[0081] In a possible embodiment, determining the time-domain dynamic features of eye movements based on the data of the saccadic latency stage includes: calculating the standard deviation of the eye acceleration within the saccadic latency stage based on the eye acceleration corresponding to each sampling time within the saccadic latency stage, and using the standard deviation of the eye acceleration as the time-domain dynamic features.

[0082] For example, assuming there are k viewpoints in the latent phase, the acceleration at the k viewpoints is recorded as , ,…… , then the standard deviation of acceleration during the latent period is ,in, is the mean eye acceleration during the latency period.

[0083] In a possible embodiment, the method further includes: using a nonlinear least squares method to fit the saccade landing point error in each trial saccade task, obtaining an error decay rate, and using the error decay rate as a time domain dynamic feature.

[0084] In one possible implementation, the formula for the landing error attenuation analysis is: ;in, represents the saccade landing error at time t after the saccade task begins, is the initial landing point error, Indicates the error decay rate and error decay time constant .

[0085] In the specific implementation, the nonlinear least squares method is used to fit the eye saccade landing error of the multi-trial eye saccade task to solve the optimal value.

[0086] In this embodiment, the error decay rate reflects the error correction mechanism of the subject's cerebellum. The landing error of healthy people decays rapidly ( The value is large), reflecting that the cerebellum's online error correction mechanism is normal, while people with health risks A decrease in the value indicates a loss of adaptive ability.

[0087] For example, for healthy people, assuming that one saccade task is performed every 30 seconds, and the test of the saccade task is performed continuously for six trials, the saccade landing point error data is obtained as follows: , , , , , obtained by this method , The same method was used to test the eye movement of Huntington's patients, and the saccade landing error data was obtained as follows: , , , , , fitting is obtained , .

[0088] In this embodiment, the error attenuation rate is obtained by fitting the landing point error, which can quantify the long-term stability of the subject. Traditionally, relying solely on the average landing point error cannot reflect the compensatory characteristics that change over time, while the landing point error attenuation constant can reveal latent functional decline and help improve the accuracy of early health monitoring.

[0089] In a possible embodiment, the method further includes: extracting frequency domain features of eye movements based on data from the saccade execution phase, specifically including the following steps:

[0090] Step 1: Calculate the instantaneous rate corresponding to each sampling moment according to the viewpoint position information corresponding to each sampling moment in the saccade execution phase, and obtain the instantaneous rate sequence of the saccade execution phase.

[0091] In this step, the calculation method of the instantaneous rate sequence refers to the above-mentioned relevant embodiments.

[0092] Step 2: Performing Hilbert transform on the instantaneous rate sequence of the saccade execution phase to obtain an analytical signal corresponding to the instantaneous rate sequence.

[0093] In this step, the Hilbert transform is applied to generate the analytical signal s(t), whose mathematical expression is:

[0094] ;

[0095] Among them, the real part of the analytical signal is the instantaneous rate sequence of the saccade execution phase, the imaginary part is the result of Hilbert transform.

[0096] Step 3: Determine the instantaneous amplitude envelope signal based on the analytical signal.

[0097] In this step, the envelope signal reflects the change of the velocity curve amplitude over time. The instantaneous amplitude envelope signal is: .

[0098] Step 4: Calculate the power spectrum density of the instantaneous amplitude envelope signal.

[0099] In this step, the envelope signal can be Fourier transformed or the power spectrum density can be calculated using the Welch method, and then the dominant frequency component can be extracted. The corresponding formula is: ;in, The main frequency of the rate fluctuation reflects the main frequency characteristics of the change of eye movement rate. is the instantaneous phase.

[0100] Step 5: Extracting frequency domain features of eye movement based on the power spectrum density; the frequency domain features include dominant frequency components.

[0101] Step 6: Generate the health reminder information based on the time domain static features, the time domain dynamic features and the frequency domain features.

[0102] In this step, the power spectral density (PSD) describes the distribution of signal power in the frequency domain. By calculating the PSD, we can find the distribution of signal power at different frequencies and extract the dominant frequency components, which are the frequency components with the largest power share in the PSD.

[0103] In one possible implementation, the calculated power spectrum density curve is used to identify one or more peaks with the highest power. The frequencies corresponding to these peaks are the frequency components with the largest power in the power spectrum density, i.e., candidate frequencies for the dominant frequency components. Specific screening criteria are set based on actual needs and signal characteristics. For example, the peak power must exceed a certain threshold (such as a certain percentage of the maximum power), or the frequency interval between adjacent peaks must meet certain conditions. The dominant frequency components are ultimately determined through screening.

[0104] For example, in saccade analysis, the dominant frequency component of the power spectral density of the saccade velocity curve of healthy people is concentrated in 8-12Hz, reflecting the normal cerebellar regulation rhythm; the dominant frequency of Alzheimer's patients is reduced to 3-5Hz, and abnormal high-frequency noise (14-18Hz) appears, suggesting that the cortex-cerebellum loop is out of sync.

[0105] In this embodiment, by mapping the instantaneous rate sequence of the saccade execution phase to the frequency domain and extracting the frequency domain features, subtle abnormalities that cannot be identified in the time domain can be captured. The frequency domain features are combined with the time domain static features and the time domain dynamic features for multi-dimensional feature analysis, which can significantly improve the accuracy of early health risk monitoring in the nervous system function.

[0106] In this embodiment, the data obtained from the eye movement test is divided into three stages according to the occurrence of saccades from the time dimension: saccade latency stage, saccade execution stage and saccade stabilization stage (refer to Figure 4 ), then, the saccades in the execution phase are divided into main saccades and corrective saccades, and the time domain static features are extracted from each of them; the time domain dynamic features (the standard deviation of eye acceleration in the latent phase and the attenuation constant of landing point error in the stable phase) are extracted based on the data of the saccade latency phase and the data of the saccade stability phase respectively; the frequency domain features are extracted based on the data of the saccade execution phase, and the integration of multi-scale and multi-dimensional features breaks through the limitations of traditional static feature analysis. Since features of different dimensions can reflect different neural states of the brain, the effectiveness and comprehensiveness of feature parameters are significantly improved from multiple dimensions such as the time domain static dimension, dynamic compensation mechanism and frequency domain, providing comprehensive and reliable data support for health monitoring of early neurological dysfunction.

[0107] In a possible embodiment, the saccade task includes one or more of a forward saccade task, a backward saccade task, and a memory saccade task.

[0108] Example, design reference for forward saccade task Figure 5As shown in the figure, each trial begins with a black fixation point (FP) appearing in the center of the screen, measuring 1 visual angle horizontally and vertically. When the FP appears, the subject is instructed to look toward it as quickly as possible. If the subject's eyes enter a detection window (the larger dashed circle in the figure) with a radius of 4 visual angles centered on the FP and remain within it for 800 milliseconds (ms), the FP disappears. Simultaneously, a solid stimulus point, measuring 1 visual angle and 10 visual angles from the center of the screen, appears randomly at one of four peripheral locations: up, down, left, or right. After the stimulus point appears, the subject is instructed to look toward it as accurately and quickly as possible and maintain fixation. If the subject's eyes enter a detection window with a radius of 4 visual angles centered on the target stimulus point and remain there for 300 ms, the stimulus point disappears and the trial ends. If the subject still does not look toward the FP after 1000 ms or does not make a saccade after 2000 ms after the target stimulus point appears, the trial ends. After each trial, there was an 800-ms intertrial period during which no visual stimulus was displayed on the screen to avoid residual vision on the retina.

[0109] Design reference for the countersaccade task Figure 6 As shown in the figure, each trial begins with a black fixation dot (FP) measuring 1 visual angle horizontally and 1 visual angle vertically, appearing in the center of the screen. When the FP appears, the subject is instructed to look toward the fixation dot as quickly as possible. If the subject's eyes enter a detection window with a radius of 4 visual angles centered on the FP and remain within that window for 800 ms, the FP disappears. Simultaneously, a solid stimulus dot, measuring 1 visual angle and 10 visual angles from the center of the screen, appears randomly at one of four peripheral locations: up, down, left, or right. After the stimulus dot appears, the subject must not look toward it. Instead, they must look as accurately and quickly as possible to a location equidistant from the stimulus dot and maintain fixation. If the subject's eyes enter a detection window with a radius of 4 visual angles centered on the point equidistant from the stimulus dot and remain within that window for 300 ms, the stimulus dot disappears and the trial ends. If the subject still does not look toward the FP after 1000 ms or does not make a saccade after 2000 ms, the trial ends. After each trial, there was an 800 ms intertrial period during which no visual stimulus was displayed on the screen.

[0110] Design reference for memory saccade task Figure 7As shown in the figure, at the beginning of each trial, a fixation point (FP) appears in the center of the screen, and the subject is required to look at the FP as quickly as possible. After the subject has fixed their gaze on the FP for the required duration, a stimulus point will randomly appear at one of four peripheral locations: up, down, left, or right. The subject is required to maintain their fixation on the FP and remember its spatial location. After 500ms, the stimulus point disappears, followed by a 1000ms delay period. During this delay period, the subject must continue to fixate on the FP. After the delay period, the FP point disappears, and the subject is required to return to the location where the stimulus point appeared as accurately and quickly as possible.

[0111] Figure 8 is an application scenario diagram of a method for extracting multi-dimensional eye movement features provided by another embodiment of the present invention; Figure 8 As shown in FIG, the scenario includes an eye tracker, a test screen, and a cloud server. The following describes another method for generating health prompt information based on this scenario.

[0112] In a possible embodiment, generating health prompt information based on the time-domain static features and the time-domain dynamic features includes: performing fully homomorphic encryption on the time-domain static features and the time-domain dynamic features based on (Ring Learning With Errors, RLWE) ciphertext to obtain encrypted time-domain static features and time-domain dynamic features; sending the encrypted time-domain static features and time-domain dynamic features to a cloud server, so that the cloud server compares the encrypted time-domain static features and time-domain dynamic features with a preset judgment threshold, and generates the health prompt information based on the comparison result.

[0113] It should be noted that fully homomorphic encryption is a privacy-enhancing technology that, while protecting transmission confidentiality through traditional cryptographic algorithms, also supports arbitrary computations on plaintext and ciphertext. The decrypted computational result is consistent with the result of direct computation on the plaintext. Therefore, eye movement features can be compared with judgment thresholds in an encrypted state without the need for decryption, thereby protecting data privacy. In this invention, the judgment thresholds are stored in a cloud server, and threshold comparisons are performed in the cloud. The comparison algorithm in the fully homomorphic encryption scheme is used to achieve end-to-end privacy protection for early monitoring of nervous system function.

[0114] In a possible implementation, assuming that the plain text value of the eye movement feature that needs privacy protection is a, and the plain text value of the judgment threshold that needs privacy protection is b, Taking the comparison judgment of as an example (the comparison algorithms of other features are similar), the input of the comparison algorithm is the RLWE ciphertext RLWE of encrypted a ( ), extract the constant term of RLWE ciphertext and convert it into LWE ciphertext. The algorithm output is LWE ciphertext LWE (1 / 0). If a and b satisfy If the comparison condition is met, the ciphertext of 1 is output, otherwise the ciphertext of 0 is output. Specifically, input RLWE( ) and then, by the formula Calculate the intermediate results , using the Extract algorithm from Extract LWE(out)=Extract(RLWE(out)). Among them, Represents the plaintext and ciphertext multiplication in the fully homomorphic encryption scheme, satisfying In polynomial ring operations, , by calculating ,when When , the constant term of the calculation result is 1 / 2, when When , the constant term is -1 / 2; Next, perform the homomorphic addition result and add the above result to Perform homomorphic addition, when When , the polynomial constant term of the final plaintext is 1 / 2+1 / 2=1; when When , its constant term is 1 / 2-1 / 2=0. Finally, the Extract algorithm is executed. This algorithm is a sample extraction algorithm in the fully homomorphic encryption scheme. The input of the Extract algorithm is the RLWE ciphertext, and the output is the LWE ciphertext of its constant term, obtaining the final LWE(1) or LWE(0).

[0115] For example, assuming N=4, a=1, b=2, and following the above calculation steps, we first get the multiplication result: , then its constant term is 1 / 2, the final result is 1 / 2+1 / 2=1, and the output is LWE(1).

[0116] In this embodiment, the terminal device adopts fully homomorphic encryption technology, and encrypts various eye movement features based on the RLWE ciphertext at the terminal to obtain the ciphertext form of the eye movement feature RLWE ( ), the ciphertext of the eye movement features is sent to the cloud server. The cloud server compares the eye movement features with the corresponding judgment threshold based on the ciphertext without decryption and generates health prompt information. On the cloud server, any attacker cannot obtain the ciphertext RLWE( ) to recover the plaintext value of the eye movement feature; the judgment threshold is stored in the cloud server, not the terminal device, making it impossible for attackers to tamper with the threshold at will, achieving end-to-end privacy protection. In addition, the judgment threshold can be dynamically updated in the cloud server without relying on firmware upgrades.

[0117] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0118] The following are device embodiments of the present invention. For details not fully described therein, reference may be made to the corresponding method embodiments described above.

[0119] Figure 9 The following is a schematic diagram showing the structure of a device for extracting multi-dimensional eye movement features according to an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0120] like Figure 9 As shown, the device for extracting multi-dimensional eye movement features includes: a data acquisition module 901, which is used to collect the subject's viewpoint position information on the test screen according to a preset sampling frequency and record the sampling time corresponding to each viewpoint position information when the subject performs an eye movement test according to a predetermined eye movement task; a data processing module 902, which is used to determine the data of the eye movement latency stage, the data of the eye movement execution stage and the data of the eye movement stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; a feature extraction module 903, which is used to determine the time domain static features of eye movement according to the data of the eye movement latency stage, the data of the eye movement execution stage and the data of the eye movement stabilization stage; and, to determine the time domain dynamic features of eye movement according to the data of the eye movement latency stage; an information generation module 904, which is used to generate health prompt information according to the time domain static features and the time domain dynamic features.

[0121] In a possible implementation, the data of the saccade latency stage include: saccade latency; the data of the saccade execution stage include: saccade amplitude, saccade duration and saccade rate; the data of the saccade stabilization stage include: saccade landing point error; the data processing module 902 is specifically used to: determine the start time and end time of the main saccade in each trial saccade task, as well as the start time and end time of each corrective saccade according to the viewpoint position information and the sampling time corresponding to each viewpoint position information; wherein the main saccade is the first saccade that occurs during the subject's scanning from the fixation point to the target position, and the corrective saccade is the saccade that occurs after the end time of the main saccade; the difference between the time when the fixation point disappears on the test screen in each trial saccade task and the start time of the main saccade is calculated. as the saccade latency of each trial saccade task; determine the saccade amplitude and saccade rate of each corrective saccade according to the viewpoint position information corresponding to the start time of each corrective saccade and the viewpoint position information corresponding to the end time of each corrective saccade; use the absolute value of the difference between the start time of each corrective saccade and the end time of each corrective saccade as the saccade duration of each corrective saccade; determine the saccade landing point error of each trial saccade task according to the viewpoint position information corresponding to the target position on the test screen in each trial saccade task and the end time of the last corrective saccade; wherein the fixation point is the center point of the test screen, and the target position is the position of the stimulus point displayed on the test screen or the position of the symmetrical point of the stimulus point relative to the center point.

[0122] In a possible implementation, the feature extraction module 903 is specifically used to: calculate the average saccade latency corresponding to the n-trial saccade task according to the saccade latency of each saccade task; wherein n is an integer greater than zero; calculate the average saccade landing point error corresponding to the n-trial saccade task according to the saccade landing point error of each saccade task; calculate the average saccade amplitude of the corrective saccades in the each-trial saccade task according to the saccade amplitude of each corrective saccade; calculate the average saccade duration of the corrective saccades in the each-trial saccade task according to the saccade duration of each corrective saccade; calculate the saccade peak rate of the corrective saccade in the each-trial saccade task according to the saccade rate of each corrective saccade; and use the average saccade latency, the average saccade amplitude, the average saccade duration, the average saccade landing point error, and the saccade peak rate as the time-domain static features.

[0123] In one possible implementation, the data of the saccadic latency stage also includes: the eye acceleration corresponding to each sampling moment in the saccadic latency stage; the data processing module 902 is further used to: calculate the instantaneous rate corresponding to each sampling time based on each viewpoint position information and the sampling time corresponding to each viewpoint position information; and determine the eye acceleration corresponding to each sampling time in the saccadic latency stage based on each sampling time and the instantaneous rate corresponding to each sampling time.

[0124] In a possible implementation, the feature extraction module 903 is further used to calculate the standard deviation of the eye acceleration within the saccade latency stage based on the eye acceleration corresponding to each sampling time within the saccade latency stage, and use the standard deviation of the eye acceleration as the time domain dynamic feature.

[0125] In one possible implementation, the data processing module 902 is further used to: calculate the instantaneous rate corresponding to each sampling moment according to the viewpoint position information corresponding to each sampling moment in the saccade execution phase, and obtain the instantaneous rate sequence of the saccade execution phase; perform Hilbert transform on the instantaneous rate sequence of the saccade execution phase to obtain the analytical signal corresponding to the instantaneous rate sequence; determine the instantaneous amplitude envelope signal according to the analytical signal; calculate the power spectrum density of the instantaneous amplitude envelope signal; the feature extraction module 903 is further used to: extract the frequency domain features of the eye movement according to the power spectrum density; the frequency domain features include the dominant frequency component; the information generation module 904 is further used to generate the health prompt information according to the time domain static features, the time domain dynamic features and the frequency domain features.

[0126] In one possible implementation, the device also includes: a data encryption module 905, which is used to perform fully homomorphic encryption on the time-domain static features and the time-domain dynamic features based on the RLWE ciphertext to obtain encrypted time-domain static features and time-domain dynamic features; and send the encrypted time-domain static features and time-domain dynamic features to a cloud server, so that the cloud server compares the encrypted time-domain static features and time-domain dynamic features with a preset judgment threshold, and generates the health prompt information based on the comparison result.

[0127] Figure 10 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 10 As shown, the electronic device 10 of this embodiment includes: a processor 100 and a memory 101. The memory 101 stores a computer program 102. When the processor 100 executes the computer program 102, the steps of the above-mentioned method embodiments are implemented. Alternatively, when the processor 100 executes the computer program 102, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0128] For example, the computer program 102 may be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 102 in the electronic device 10.

[0129] The electronic device 10 may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will appreciate that Figure 10 This is merely an example of the electronic device 10 and does not constitute a limitation of the electronic device 10. The electronic device 10 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 10 may also include input and output devices, network access devices, buses, etc.

[0130] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0131] The memory 101 may be an internal storage unit of the electronic device 10, such as the hard disk or memory of the electronic device 10. The memory 101 may also be an external storage device of the electronic device 10, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 10. Furthermore, the memory 101 may include both an internal storage unit of the electronic device 10 and an external storage device. The memory 101 is used to store the computer program 102 and other programs and data required by the electronic device 10. The memory 101 may also be used to temporarily store data that has been output or is about to be output.

[0132] For the sake of convenience and brevity, the division of the above functional modules / units is only used as an example. In actual applications, the above functions can be assigned to different functional modules / units as needed. The above modules / units can be implemented in the form of hardware, software, or a combination of hardware and software.

[0133] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods in the above-mentioned method embodiments.

[0134] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the methods in the above-mentioned method embodiments.

[0135] The term "computer program" includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. Computer-readable media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunications signals, and software distribution media.

[0136] In the above embodiments, the descriptions of each embodiment have their own focus. For parts not described or recorded in detail in one embodiment, please refer to the relevant descriptions of other embodiments. Unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other. The technical features of different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0137] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A method for extracting multi-dimensional eye movement features, characterized in that: include: When the subject performs an eye movement test according to a predetermined saccade task, the subject's viewpoint position information on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded; Determining data of a saccade latency phase, a saccade execution phase, and a saccade stabilization phase according to each viewpoint position information and a sampling time corresponding to each viewpoint position information; The data of the saccade execution phase includes instantaneous rate sequence; the data of the saccade stabilization phase includes the saccade landing point error of each saccade task; Determining the temporal static characteristics of the eye movement based on the data of the saccade latency stage, the data of the saccade execution stage, and the data of the saccade stabilization stage; and determining the temporal dynamic characteristics of the eye movement based on the data of the saccade latency stage and the data of the saccade stabilization stage respectively; The temporal dynamic features include the standard deviation of eye acceleration during the saccade latency phase and the error decay rate during the saccade stabilization phase; the error decay rate is determined based on the saccade landing point error of each saccade task; Extracting frequency domain features of eye movements based on data from the saccade execution phase; the frequency domain features include dominant frequency components; The dominant frequency component is obtained by mapping the instantaneous rate sequence into the frequency domain to obtain a corresponding power spectrum density and extracting it from the power spectrum density; Health reminder information is generated based on the time domain static features, the time domain dynamic features and the frequency domain features.

2. The method for extracting multi-dimensional eye movement features according to claim 1, characterized in that: The data of the saccade latency stage include: saccade latency; the data of the saccade execution stage include: saccade amplitude, saccade duration and saccade rate; The method of determining data of the saccade latency stage, the saccade execution stage, and the saccade stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information comprises: Determining the start time and end time of the main saccade in each saccade task, as well as the start time and end time of each corrective saccade, based on the viewpoint position information and the sampling time corresponding to each viewpoint position information; wherein the main saccade is the first saccade that occurs during the subject's scanning process from the fixation point to the target position, and the corrective saccade is the saccade that occurs after the end time of the main saccade; The absolute value of the difference between the disappearance time of the fixation point on the test screen and the start time of the main saccade in each trial of the saccade task is used as the saccade latency of each trial of the saccade task; Determining the saccade amplitude and saccade rate of each corrective saccade according to the viewpoint position information corresponding to the start time of each corrective saccade and the viewpoint position information corresponding to the end time of each corrective saccade; The absolute value of the difference between the start time of each corrective saccade and the end time of each corrective saccade is used as the saccade duration of each corrective saccade; The saccade landing error of each saccade task was determined based on the viewpoint position information corresponding to the target position on the test screen and the end time of the last corrective saccade in each saccade task. The fixation point is the center point of the test screen, and the target position is the position of the stimulation point displayed on the test screen or the position of the symmetrical point of the stimulation point relative to the center point.

3. The method for extracting multi-dimensional eye movement features according to claim 2, characterized in that: Determining the temporal static features of the eye movement according to the data of the saccade latency stage, the data of the saccade execution stage, and the data of the saccade stabilization stage includes: Calculating the average saccade latency corresponding to n saccade tasks based on the saccade latency of each saccade task trial; wherein n is an integer greater than zero; Calculate the average saccade landing error corresponding to the n-trial saccade task according to the saccade landing error of each trial saccade task; Calculating the average saccade amplitude of the corrective saccades in each trial of the saccade task according to the saccade amplitude of each corrective saccade; Calculating the average saccade duration of the corrective saccades in each trial of the saccade task according to the saccade duration of each corrective saccade; Calculating the peak rate of the corrective saccade in each trial of the saccade task according to the saccade rate of each corrective saccade; The average saccade latency, the average saccade amplitude, the average saccade duration, the average saccade landing point error and the saccade peak rate are used as the time-domain static features.

4. The method for extracting multi-dimensional eye movement features according to claim 2, wherein: The data of the saccadic latency stage also includes: eye acceleration corresponding to each sampling moment in the saccadic latency stage; Determining data of the saccadic latency stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information, further comprising: Calculating the instantaneous rate corresponding to each sampling time according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; The eye acceleration corresponding to each sampling time in the saccadic latency stage is determined according to each sampling time and the instantaneous rate corresponding to each sampling time.

5. The method for extracting multi-dimensional eye movement features according to claim 4, characterized in that: Determining the temporal dynamic characteristics of eye movements based on the data of the saccadic latency stage includes: According to the eye acceleration corresponding to each sampling time in the saccadic latency stage, the standard deviation of the eye acceleration in the saccadic latency stage is calculated, and the standard deviation of the eye acceleration is used as the time domain dynamic feature.

6. The method for extracting multi-dimensional eye movement features according to claim 1, characterized in that: Extracting frequency domain features of eye movements based on data from the saccade execution phase includes: Calculating the instantaneous rate corresponding to each sampling moment according to viewpoint position information corresponding to each sampling moment in the saccade execution phase to obtain an instantaneous rate sequence of the saccade execution phase; performing a Hilbert transform on the instantaneous rate sequence of the saccade execution phase to obtain an analytical signal corresponding to the instantaneous rate sequence; Determining an instantaneous amplitude envelope signal according to the analytical signal; Calculating the power spectral density of the instantaneous amplitude envelope signal; The frequency domain features of the eye movement are extracted according to the power spectrum density; the frequency domain features include a dominant frequency component.

7. The method for extracting multi-dimensional eye movement features according to any one of claims 1 to 6, characterized in that: The saccade task includes one or more of a forward saccade task, a backward saccade task, and a memory saccade task.

8. The method for extracting multi-dimensional eye movement features according to any one of claims 1 to 6, characterized in that: Health reminder information may also be generated based on the time domain static features and the time domain dynamic features, including: Performing fully homomorphic encryption on the time-domain static features and the time-domain dynamic features based on the RLWE ciphertext to obtain encrypted time-domain static features and time-domain dynamic features; The encrypted time-domain static features and time-domain dynamic features are sent to a cloud server, so that the cloud server compares the encrypted time-domain static features and time-domain dynamic features with a preset judgment threshold, and generates the health prompt information based on the comparison result.

9. A device for extracting multi-dimensional eye movement features, characterized in that: include: The data acquisition module is used to collect the subject's viewpoint position information on the test screen according to a preset sampling frequency and record the sampling time corresponding to each viewpoint position information when the subject performs an eye movement test according to a predetermined eye movement task. A data processing module is used to determine data of a saccade latency stage, data of a saccade execution stage, and data of a saccade stabilization stage according to each viewpoint position information and a sampling time corresponding to each viewpoint position information; The data of the saccade execution phase includes instantaneous rate sequence; the data of the saccade stabilization phase includes the saccade landing point error of each saccade task; A feature extraction module is used to determine the time-domain static features of the eye movement based on the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and determining the temporal dynamic characteristics of the eye movement based on the data of the saccade latency stage and the data of the saccade stabilization stage respectively; The time domain dynamic features include the standard deviation of eye acceleration in the saccade latency phase and the error decay rate in the saccade stabilization phase; the error decay rate is determined based on the saccade landing point error of each saccade task; the frequency domain features of the eye movement are extracted based on the data of the saccade execution phase; the frequency domain features include the dominant frequency component; The dominant frequency component is obtained by mapping the instantaneous rate sequence into the frequency domain to obtain a corresponding power spectrum density and extracting it from the power spectrum density; The information generation module is used to generate health reminder information based on the time domain static features, the time domain dynamic features and the frequency domain features.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

Images