Multi-dimensional eye movement feature extraction method, device and equipment

Abstract

The invention provides a multi-dimensional eye movement feature extraction method, device and equipment. Relates to the technical field of data processing, and the method comprises the following steps: when a subject performs an eye movement test according to a predetermined eye jump task, collecting viewpoint position information of the subject on a test screen according to a preset sampling frequency, and recording sampling time corresponding to each piece of viewpoint position information; determining data of an eye jump incubation stage, an execution stage and a stable stage according to the position information of each viewpoint and the sampling time; determining time domain static characteristics of eye movement according to the data of the incubation stage, the execution stage and the stable stage of the eye jump; determining time domain dynamic characteristics of eye movement according to the data of the eye jump incubation period stage; and generating health prompt information according to the time domain static features and the time domain dynamic features. According to the invention, accurate, effective and comprehensive multi-dimensional eye movement features can be extracted from the multi-time-scale eye movement data, so that the health prompt information generated for the subject is more accurate and comprehensive.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and equipment for extracting multi-dimensional eye movement features. Background Art

[0002] Eye movement test is a test method that tracks physiological parameters such as eye movement trajectory, fixation point, and movement speed of the subject on the test screen. Since the process of eye movement is coordinated and controlled by multiple areas of the brain (including the frontal lobe eye movement area, parietal lobe, basal ganglia, thalamus, brainstem and other structures). Therefore, the results of eye movement test can reflect specific brain functions, such as attention allocation, visual information processing, emotional state, memory strength, inhibitory control ability and other cognitive conditions and physiological states. Therefore, the eye movement data obtained from the eye movement test can reveal whether the subject has potential health risks in terms of cognition, emotion and nervous system function.

[0003] In related technologies, the method of obtaining and processing eye movement data is usually to present a fixation point on the test screen, requiring the subject to look at the fixation point first, and then some stimulation points appear, requiring the subject to look from the fixation point to the stimulation point, and using an eye tracker to track the subject's eye position, movement trajectory and other eye movement data during the test. The eye tracker pre-stores a judgment threshold, compares the collected eye movement data with the corresponding judgment threshold, predicts whether the subject has health risks based on the comparison results, and sends health warning information to the subject.

[0004] However, traditional methods can only rely on eye trackers to obtain single-dimensional eye movement data. In the actual eye movement test process, the subjects are easily disturbed by various factors such as ambient lighting and eye health, resulting in inaccurate eye movement data collected. Therefore, single-dimensional eye movement data cannot accurately reflect the subjects' real cognitive changes and physiological states, making it difficult to achieve accurate predictions and resulting in a high error rate, which leads to inaccurate health warning information sent. Summary of the invention

[0005] The embodiments of the present invention provide a method, device and equipment for extracting multi-dimensional eye movement features to solve the problem that single-dimensional eye movement data in the prior art cannot accurately reflect the real cognitive changes and physiological state of the subject, is difficult to achieve accurate prediction, has a high misjudgment rate, and thus leads to inaccurate health warning information sent.

[0006] In a first aspect, an embodiment of the present invention provides a method for extracting multi-dimensional eye movement features, comprising: When the subject performs an eye movement test according to a predetermined eye movement task, the viewpoint position information of the subject on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded; Determine data of the saccade latency stage, data of the saccade execution stage, and data of the saccade stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; Determining the temporal static features of eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and determining the temporal dynamic features of eye movement according to the data of the saccade latency stage; Health reminder information is generated according to the time domain static features and the time domain dynamic features.

[0007] In a second aspect, an embodiment of the present invention provides a device for extracting multi-dimensional eye movement features, characterized in that it includes: a data acquisition module, which is used to collect the subject's viewpoint position information on the test screen according to a preset sampling frequency and record the sampling time corresponding to each viewpoint position information when the subject performs an eye movement test according to a predetermined eye movement task; a data processing module, which is used to determine the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; a feature extraction module, which is used to determine the time domain static features of eye movements according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and, determine the time domain dynamic features of eye movements according to the data of the saccade latency stage; an information generation module, which is used to generate health prompt information according to the time domain static features and the time domain dynamic features.

[0008] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method in the first aspect or any possible implementation of the first aspect is implemented.

[0009] In an embodiment of the present invention, when the subject performs an eye movement test according to a predetermined eye movement task, the viewpoint position information of the subject on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded; according to each viewpoint position information and the sampling time corresponding to each viewpoint position information, the data of the eye movement latency stage, the data of the eye movement execution stage and the eye movement stabilization stage are determined; according to the data of the eye movement latency stage, the data of the eye movement execution stage and the eye movement stabilization stage, the time domain static characteristics of the eye movement are determined; and, according to the data of the eye movement latency stage, the time domain dynamic characteristics of the eye movement are determined; and health prompt information is generated according to the time domain static characteristics and the time domain dynamic characteristics. The embodiment of the present invention divides the entire eye movement process into multiple scales from the time dimension, and determines the eye movement data of the three stages of eye movement latency stage, execution stage and stabilization stage respectively. This division method is closer to the physiological mechanism of eye movement, and then extracts the time domain static features of eye movement based on the data of the three stages, and extracts the dynamic features for the latent stage. The effectiveness, accuracy and comprehensiveness of the eye movement features are improved based on the multi-dimensional eye movement features of time domain static and dynamic, so that a more comprehensive health assessment can be performed on the subjects, and the health prompt information finally generated is also more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is an application scenario diagram of the method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention; Figure 2 is a flow chart of an implementation of a method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention; Figure 3 is a schematic diagram of test screen coordinate conversion provided by an embodiment of the present invention; Figure 4 is a schematic diagram of a multi-dimensional feature extraction architecture provided by an embodiment of the present invention; Figure 5 is a schematic diagram of a forward saccade task design provided by an embodiment of the present invention; Figure 6 is a schematic diagram of the reverse saccade task design provided by an embodiment of the present invention; Figure 7 is a schematic diagram of a design of a memory saccade task provided by an embodiment of the present invention; Figure 8 is an application scenario diagram of a method for extracting multi-dimensional eye movement features provided by another embodiment of the present invention; Fig. 9 is a schematic diagram of the structure of a device for extracting multi-dimensional eye movement features provided by an embodiment of the present invention; Fig.10 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] Based on the defects of the prior art, this application proposes the following technical ideas: divide the entire eye movement process into multiple scales from the time dimension, and determine the eye movement data of the three stages of the saccade latency stage, execution stage and stabilization stage respectively. Since the regulatory areas of the brain corresponding to different stages are different, this division method is closer to the physiological mechanism of eye movement. Then, the time domain static features of eye movement are extracted by combining the data of the three stages, and the dynamic features are extracted for the latent stage. According to the multi-dimensional eye movement features of time domain static and dynamic, it is judged whether the subject has potential health risks. Through stage division and multi-dimensional feature extraction, the effectiveness, accuracy and comprehensiveness of the features can be improved, and a more comprehensive health assessment of the subjects can be carried out, and the health prompt information finally generated is also more accurate.

[0013] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0014] Figure 1 A diagram of an application scenario of the method for extracting multi-dimensional eye movement features provided in an embodiment of the present invention.

[0015] like Figure 1 As shown, the application scenario provided by this embodiment includes: an eye tracker 11 and a test screen 12; wherein the test screen 12 can be, but is not limited to, a screen of a display that communicates with a server.

[0016] The subject wears an eye tracker 11 and quickly looks at the fixation point (FP) at the center of the test screen 12 at the beginning of the eye movement test. Then, when the fixation point FP disappears, a stimulation point appears on the test screen, and the subject needs to quickly look at the stimulation point as required. In this process, the eye tracker collects the subject's eye image according to a certain sampling frequency domain, and then processes the eye image to obtain the subject's viewpoint position on the test screen 12. The viewpoint position sequence corresponding to all sampling moments constitutes an eye movement trajectory of the eye movement test process.

[0017] It is understandable that the application scenario provided in this embodiment is only used as an example to facilitate relevant personnel to understand the method provided by the present invention, and does not constitute any limitation to the present invention.

[0018] The method provided by the present invention will be described in detail below in conjunction with application scenarios.

[0019] See also Figure 2 , which shows a flowchart of the implementation of the method for extracting multi-dimensional eye movement features provided by an embodiment of the present invention, and the detailed steps are as follows: S201, when a subject performs an eye movement test according to a predetermined eye movement task, the viewpoint position information of the subject on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded.

[0020] In this step, saccades can be divided into reflexive saccades and autonomous saccades according to the driving factors of saccades, and the task paradigm of the corresponding judgment test can be designed according to the driving factors of saccades.

[0021] In one possible implementation, a progressive saccade task (PS) is designed to respond to the subject's reflexive saccades. At the beginning of the trial, the fixation point FP is displayed in the center of the test screen. The subject needs to look at the fixation point FP as quickly as possible and keep looking at it until the time requirement is met. Then, a stimulus point randomly appears outside the fixation point FP, and the subject needs to look at the stimulus point quickly.

[0022] In one possible implementation, an anti-saccade task (AS) is designed to test the subject's autonomous eye movements. In the anti-saccade task AS, the subject needs to look at the fixation point FP as quickly as possible and keep looking until the time requirement is met. Then, a stimulus point appears randomly outside the fixation point FP, and the subject needs to quickly look at the target position opposite to the stimulus point (i.e., the position of the symmetrical point of the stimulus point relative to the fixation point).

[0023] In one possible implementation, a memory-guided saccade task (MGS) is designed to test the subject's autonomous saccades. At the beginning of the trial, the test screen displays the fixation point FP, and then stimulus points are randomly displayed outside the fixation point. The stimulus points then disappear. At this time, the subject still keeps looking at the fixation point FP. During the delay period, the subject needs to remember the position of the stimulus point and quickly look at the target position where the stimulus point appeared when the fixation point FP disappears.

[0024] It should be noted that the detailed processes of the above three eye-saccade tasks will be further described in the following embodiments.

[0025] In this step, the preset sampling frequency is at least 1000 times / second. The sampling device can be, but is not limited to, an eye tracker based on infrared video. The viewpoint position information is the visual angle coordinates of the viewpoint relative to the center of the test screen.

[0026] In a possible implementation, when a subject performs an eye movement test according to a predetermined eye movement task, an infrared video-based eye tracker illuminates the subject's eye area with infrared light, and a camera built into the eye tracker records eye images. From the beginning of a trial eye movement task to the end of the trial eye movement task, a set of eye images is collected, and then each frame of the eye image set is processed using a computer vision algorithm to obtain the subject's eye features, including pupil center coordinates, pupil size, and corneal reflection point coordinates, etc. Based on the above eye features and the relative position relationship between the eye and the test screen, the viewpoint coordinates are calculated, and the viewpoint coordinates are pixel coordinates. Then, the pixel coordinates of each viewpoint are converted into the visual angle coordinates of the viewpoint relative to the center of the test screen, that is, the viewpoint position information is obtained, and the sampling time corresponding to each viewpoint position information is recorded to obtain a position-time series.

[0027] Exemplary, reference Figure 3 , the resolution of the test screen is 1920*1080 pixels. The coordinate system of the test screen is established with the upper left corner of the test screen as the origin. The pixel coordinates of the upper left corner are (0, 0), the pixel coordinates of the lower right corner are (1920, 1080), the pixel coordinates of the center of the screen are (960, 540), and the pixel coordinates of the viewpoint are (eyeX, eyeY). The visual angle coordinates corresponding to the viewpoint are ( , ), pdd represents the number of pixels corresponding to each viewing angle; the value of pdd needs to be determined based on the distance between the subject's eyes and the screen when the subject is actually conducting an eye movement test.

[0028] In this step, the sampling frequency is set to at least 1000 times / second. The high sampling frequency can capture subtle changes in the eyes and help extract high-dimensional eye movement features. In addition, in eye movement research, pixel coordinates are converted into visual angle coordinates. The use of viewing angle can more accurately describe the movement and gaze position of the eyes, which is not affected by the screen resolution or the size of the display device, and can more intuitively reflect the eyes' attention to information at different positions.

[0029] S202, determining data of a saccade latency stage, data of a saccade execution stage, and data of a saccade stabilization stage according to each viewpoint position information and a sampling time corresponding to each viewpoint position information.

[0030] In this step, the saccade latency stage is the stage between the disappearance of the fixation point and the onset of the main saccade. The latency stage reflects the brain's processing of visual information, decision-making, and the generation of motor commands, and reflects the neural efficiency of the brain's transformation from perception to movement, which belongs to the preparation stage. The saccade execution stage is the stage from the onset of the main saccade to the rapid movement of the eyeball to the target position. It belongs to the rapid saccade stage, which reflects the brain's control over eye movement. The saccade stabilization stage is the stage in which the eyeball maintains a stable fixation on the target position after the last saccade is completed. During this stage, the brain needs to maintain continuous attention to the target position and inhibit reflex saccades caused by irrelevant stimuli.

[0031] According to the position-time sequence obtained in step S201 (ie, each viewpoint position and the sampling time corresponding to the viewpoint position), data of the three stages are determined respectively according to the three divided stages.

[0032] In a possible implementation, the data of the saccade latency stage includes: saccade latency; the data of the saccade execution stage includes: saccade amplitude, saccade duration and saccade rate; the data of the saccade stability stage includes: saccade landing point error; It should be noted that the detailed calculation method of the data at each stage will be described in detail in the following embodiments.

[0033] S203, determining the time-domain static features of the eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and determining the time-domain dynamic features of the eye movement according to the data of the saccade latency stage.

[0034] In this step, in addition to determining the time domain static features based on the data of the three stages, the time domain dynamic features are extracted for the data of the incubation period. In health monitoring, dynamic features can capture subtle abnormalities from a higher dimension that traditional static features cannot reflect.

[0035] In a possible implementation, the time domain static features may include: average saccade latency, average saccade amplitude, average saccade duration, average saccade landing error, and saccade peak rate. Since a single test data may have occasional anomalies caused by environmental factors, in this embodiment, the average is taken for each data. Through these static features, it is possible to comprehensively judge whether the subject has potential health risks. For example, an extended latency may indicate degeneration of the prefrontal cortex and basal ganglia, leading to delayed saccade initiation. These static features can capture early abnormalities of the nervous system by quantifying the initiation efficiency, movement trajectory, accuracy, and speed of saccades.

[0036] Since the duration and stability of the latent stage can be used as important indicators for studying cognitive function and nervous system disorders, extracting high-dimensional features of the latent stage can help reveal the decline of the subject's early cognitive ability. Stability can be measured by the standard deviation of eye acceleration during the latent stage. Therefore, in a possible implementation, the time domain dynamic features may include: standard deviation of eye acceleration. Eye acceleration reflects the rate of change of eye movement speed during the latent stage. The standard deviation of eye acceleration quantifies the degree of discreteness of eye acceleration in the time dimension, which can reflect the stability of the subject during the latent preparation stage. In health monitoring, the standard deviation of eye acceleration during the latent stage of saccades essentially reflects the dynamic stability of the neuromuscular system during the initiation of saccades. Its increase may indicate central control disorder, increased noise in motor planning, or potential pathological states. It can be used as an auxiliary indicator for evaluating nervous system diseases in clinical health monitoring. For normal subjects, they continue to pay attention to the fixation point during the latent stage. In theory, the eyeballs will not move rapidly. Therefore, the eye acceleration of healthy subjects is very small. If the standard deviation of the subject's eye acceleration during the latent stage is large, it means that the subject's eye acceleration fluctuates greatly during the latent stage, and there may be potential problems with the neural control mechanism during the movement preparation stage. For example, the subject is anxious and has difficulty maintaining concentration, or the basal ganglia of the brain are abnormally regulated, causing the subject to be unable to accurately control eye movements.

[0037] S204, generating health reminder information according to the time domain static features and the time domain dynamic features.

[0038] In this step, after obtaining the time domain static features and the time domain dynamic features, each feature can be compared with the corresponding judgment threshold, and the comparison results of each feature can be combined to generate targeted health prompt information for the subject, which can more accurately and comprehensively evaluate the subject's nervous system function.

[0039] For example, the normal range of standard deviation of latency acceleration in healthy adults is , the judgment threshold of this feature can be set to If the standard deviation of the subject's eye acceleration is greater than the threshold and the peak rate of saccades is lower than the threshold, a health prompt message indicating damage to the basal ganglia-cortex circuit is generated.

[0040] In this embodiment, the entire eye movement process is divided into multiple scales from the time dimension, and the eye movement data of the three stages, namely the saccade latency stage, the execution stage and the stabilization stage, are determined respectively. This division method is closer to the physiological mechanism of eye movement. Then, the time domain static features of the eye movement are extracted by combining the data of the three stages, and the dynamic features are extracted for the latency stage. The time domain static and dynamic multi-dimensional eye movement features are used to judge whether the subject has potential health risks. The static features can capture subtle abnormal changes in nervous system function before the clinical symptoms of neurodegenerative diseases appear by quantifying the initiation efficiency, movement trajectory, accuracy and speed of saccades. The dynamic features make up for the defects of the static features and capture subtle abnormalities that the static features cannot reflect from a higher dimension. The combination of static and dynamic multi-dimensional features can conduct a more comprehensive health assessment of the subject, and the health prompt information finally generated is more accurate.

[0041] In a possible embodiment, the data of the saccade latency stage include: saccade latency; the data of the saccade execution stage include: saccade amplitude, saccade duration and saccade rate; the data of the saccade stability stage include: saccade landing point error. The data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stability stage are determined according to each viewpoint position information and the sampling time corresponding to each viewpoint position information, including: determining the start time and end time of the main saccade in each trial saccade task, and the start time and end time of each corrective saccade according to the viewpoint position information and the sampling time corresponding to each viewpoint position information; wherein the main saccade is the first saccade that occurs during the subject's scanning from the fixation point to the target position, and the corrective saccade is the saccade that occurs after the end time of the main saccade; the absolute value of the difference between the time when the fixation point on the test screen disappears in each trial saccade task and the start time of the main saccade is used as the saccade of each trial saccade task. incubation period; determine the saccade amplitude and saccade rate of each corrective saccade according to the viewpoint position information corresponding to the start time of each corrective saccade and the viewpoint position information corresponding to the end time of each corrective saccade; take the absolute value of the difference between the start time of each corrective saccade and the end time of each corrective saccade as the saccade duration of each corrective saccade; determine the saccade landing point error of each trial saccade task according to the viewpoint position information corresponding to the target position on the test screen in each trial saccade task and the end time of the last corrective saccade; wherein the fixation point is the center point of the test screen, and the target position is the position of the stimulus point displayed on the test screen or the position of the symmetrical point of the stimulus point relative to the center point.

[0042] It should be noted that the research on the related parameters of eye saccades in the traditional way is mostly for the main eye saccade, that is, after the disappearance of the fixation point and the appearance of the stimulus point, the first eye saccade that occurs in the process of scanning the target position from the fixation point. Under normal circumstances, eye saccade is a fast and accurate eye movement that helps us to transfer our sight to the target position. If the first eye saccade does not accurately reach the target position, corrective eye saccade will be produced. However, when the experimenter is carrying out the eye saccade task test, it is easy to be disturbed by various factors such as environmental factors and human factors, causing the main eye saccade to be abnormal, and multiple corrective eye saccades will occur afterwards. In this case, the related parameters of the main eye saccade are analyzed separately and are difficult to fully reflect the real state of the experimenter. Therefore, the present embodiment, in addition to studying the related parameters of the main eye saccade, also focuses on the related parameters of the corrective eye saccade, with the real state of the experimenter's analysis comprehensively and accurately.

[0043] It should be noted that in this embodiment, the fixation point is the center point of the test screen, and the target position is the position of the stimulus point displayed on the test screen or the position of the symmetrical point of the stimulus point relative to the center point. Specifically, when the saccade task is a forward saccade task, the target position is the position of the stimulus point; when the saccade task is a reverse saccade task, the target position is the symmetrical position of the stimulus point relative to the fixation point; when the saccade task is a memory saccade task, the target position is the position where the stimulus point appeared.

[0044] In the actual saccade task test scenario, after the fixation point disappears, the subject's main saccade landing point may have a larger error than the target position, and then multiple corrective saccades will occur within a certain period of time to reduce the landing error. The reason for the large error in the main saccade landing point may be that the subject has health risks, it may be due to head rotation, or it may be interfered by external environmental factors. For healthy subjects, when the main saccade landing point error is large, the line of sight will be quickly adjusted to the target position, and corrective saccades will occur during this viewpoint correction process; for subjects who may have early nervous system dysfunction, when the main saccade landing point error is large, the viewpoint will also be adjusted, and corrective saccades will occur, but compared with healthy subjects, the speed and time of adjusting the viewpoint position and the landing point error after adjusting the viewpoint will also be different from those of healthy people. The abnormality of corrective saccades is essentially the brain's compensatory response to eye movement errors, and changes in its frequency or pattern can be used as a sign of nervous system dysfunction. For example, abnormalities in the motor initiation and regulation of the basal ganglia in neurological disorders such as Parkinson's and Huntington's will cause delayed initiation of saccades and abnormal amplitude, resulting in an increase in corrective saccades and unstable saccade amplitude. For another example, the atrophy of the cerebral cortex (especially the prefrontal lobe and parietal lobe) in Alzheimer's disease leads to damage to the eye movement control network. The number of corrective saccades in saccade tasks increases, and the duration of corrective saccades increases and the peak rate decreases, reflecting a decrease in target positioning and motor planning ability. Therefore, in this embodiment, by extracting the characteristics of corrective saccades (saccade amplitude, rate and duration of corrective saccades), it can assist in monitoring early abnormalities in nervous system function and provide an important basis for revealing abnormal brain function.

[0045] In a possible implementation, the eye saccade detection method may adopt the following steps: Step 1: According to the viewpoint position information and the sampling time corresponding to each viewpoint position information, calculate the instantaneous rate corresponding to each sampling time and obtain the rate-time series (denoted as ).in, represents the sampling time, and i represents the number of samples.

[0046] In a possible implementation, the instantaneous rate of the viewpoint position corresponding to each sampling time is calculated based on the adjacent difference method. Specifically, the viewpoint position-time series is expressed as ( ), represents the horizontal viewing angle of the viewpoint corresponding to the i-th sampling time, represents the vertical viewing angle of the viewpoint corresponding to the i-th sampling time. Then the instantaneous speed of each viewpoint in the horizontal direction is: , the instantaneous velocity in the vertical direction is: , Represents the sampling time interval. The instantaneous rate at the viewpoint position corresponding to each sampling time is: .

[0047] Step 2: Set a speed threshold, which can be in the range of 15° / s~30° / s. Continuous viewpoints that are greater than the speed threshold for more than 20 milliseconds are determined as eye movement segments, and the first point is the starting point of the eye movement, and the last point is the ending point of the eye movement.

[0048] Step 3: Calculate the amplitude between the start and end points of each saccade segment.

[0049] Step 4: The first saccade segment in the process of scanning towards the target position after the fixation point disappears is defined as the main saccade, and the saccade segment with an amplitude greater than 2 visual angles after the main saccade is defined as the corrective saccade.

[0050] In one possible implementation, assuming that the time when the fixation point on the test screen disappears in each saccade task is t1, and the start time of the main saccade is t2, then the saccade latency of each saccade task is .

[0051] In a possible implementation, the start time of each corrective saccade is t3, and the corresponding viewpoint position information is recorded as ( ); The end time of each corrective saccade is t4, and the corresponding viewpoint position information is recorded as ( ), then the duration of each corrective saccade is T = ; The amplitude of each corrective saccade ,in, , ; Eye rate of each corrective saccade ,in, , .

[0052] In one possible implementation, the target position on the test screen in each saccade task is recorded as ( ), the viewpoint position information corresponding to the end time of the last corrective saccade is recorded as ( ), then the saccade landing error of each trial saccade task .

[0053] In a possible embodiment, determining the time-domain static features of eye movement according to the data of the saccade latency stage, the data of the saccade execution stage, and the data of the saccade stabilization stage includes: According to the saccadic latency of each trial saccadic task, the average saccadic latency corresponding to the n-trial saccadic task is calculated; wherein n is an integer greater than zero. The average saccadic latency corresponding to the n-trial saccadic task .

[0054] According to the eye saccade landing error of each eye saccade task, the average eye saccade landing error corresponding to the n-trial eye saccade task is calculated. The average eye saccade landing error corresponding to the n-trial eye saccade task .

[0055] According to the saccade amplitude of each corrective saccade, the average saccade amplitude of the corrective saccade in each trial saccade task is calculated; the average saccade amplitude , where m is the number of corrective saccades in each trial of the saccade task.

[0056] According to the saccade duration of each corrective saccade, the average saccade duration of the corrective saccade in each trial saccade task is calculated; the average saccade duration .

[0057] According to the saccade rate of each corrective saccade, the peak saccade rate of the corrective saccade in each trial saccade task is calculated; the peak saccade rate is the maximum rate among multiple corrective saccades.

[0058] In this embodiment, the core meaning of the average saccade latency is the time interval from the appearance of visual stimulation to the initiation of saccades, which reflects the efficiency of neural signal conduction, decision initiation and motor planning. It can monitor early health risks in neurodegenerative diseases. For example, a prolonged latency may indicate degeneration of the prefrontal cortex and basal ganglia, leading to delayed saccade initiation (studies show that the latency of early Alzheimer's patients is 15%-20% longer than that of healthy people; Parkinson's patients have dopaminergic neuron damage, and the efficiency of neural signal integration in the saccade preparation stage is reduced, and the latency can be extended by 10%-15%, etc.). The core meaning of the average saccade amplitude is the spatial displacement amplitude of the saccade, which reflects the ability of visual target positioning and the coordination of the eye movement system, and can reflect the early motor control disorder of the nervous system function; the average saccade duration is related to the saccade rate, reflecting the dynamic characteristics of eye movement. In the early stages of neurodegenerative diseases, the duration of saccades will be prolonged (for example, the duration of saccades in Alzheimer's disease is 10%-15% longer than that in healthy people), indicating that the extraocular muscle movement control signal is attenuated, or it is related to the functional degeneration of the basal ganglia-cerebellum loop. The average saccade landing point error reflects the visual feedback regulation and movement correction ability. A large landing point error indicates an abnormality in the brain regulation system. The peak rate of saccades reflects the explosiveness of eye movement, which depends on the contraction force of the extraocular muscles and the strength of neural drive. For example, the peak rate of Parkinson's patients will be reduced in the early stages due to dopamine deficiency.

[0059] In this embodiment, by dividing the primary saccade and the corrective saccade and focusing on the characteristics of the corrective saccade, compared with the traditional method of only studying the characteristics of the primary saccade, more subtle functional abnormalities in the early stages of nervous system dysfunction can be captured, making up for the defect that the characteristics of the primary saccade cannot fully reflect the true state of the subject, and improving the accuracy of early health problem monitoring.

[0060] In a possible embodiment, the data of the saccadic latency stage further includes: eyeball acceleration corresponding to each sampling moment in the saccadic latency stage; determining the data of the saccadic latency stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information, further includes: Calculate the instantaneous rate corresponding to each sampling time according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; According to each sampling time and the instantaneous rate corresponding to each sampling time, the eyeball acceleration corresponding to each sampling time in the saccadic latency stage is determined.

[0061] It should be noted that the calculation method of the instantaneous rate corresponding to each sampling time can refer to the introduction in the above embodiment. The instantaneous rate at the viewpoint position corresponding to each sampling time is: . Then the horizontal component of acceleration at each viewpoint is , the vertical component of acceleration is: , and finally the total acceleration at each viewpoint is According to the time of the latency stage in each trial of the saccade task, the acceleration sequence of the latency stage is extracted.

[0062] In a possible embodiment, determining the time domain dynamic features of eye movements based on the data of the saccade latency stage includes: calculating the standard deviation of the eye acceleration within the saccade latency stage based on the eye acceleration corresponding to each sampling time within the saccade latency stage, and using the standard deviation of the eye acceleration as the time domain dynamic features.

[0063] For example, assuming that there are k viewpoints in the latent stage, the acceleration at the k viewpoints is recorded as , ,…… , then the standard deviation of acceleration during the latent period is ,in, is the mean eye acceleration during the latency period.

[0064] In a possible embodiment, the method further includes: for the saccade landing point error in each trial saccade task, using a nonlinear least squares method to fit the saccade landing point error of a multi-trial saccade task to obtain an error decay rate, and using the error decay rate as a time domain dynamic feature.

[0065] In one possible implementation, the formula for the drop point error attenuation analysis is: ;in, represents the saccade landing error at time t after the saccade task begins, is the initial landing point error, Indicates the error decay rate and error decay time constant .

[0066] In the specific implementation, the nonlinear least squares method is used to fit the eye saccade landing error of the multi-trial eye saccade task to solve the optimal value.

[0067] In this embodiment, the error decay rate reflects the error correction mechanism of the subject's cerebellum. The landing error of healthy people decays rapidly ( The value is large), reflecting that the cerebellum's online error correction mechanism is normal, while people with health risks A decrease in the value indicates a loss of adaptive ability.

[0068] For example, for healthy people, assuming that one saccade task is performed every 30 seconds, and six saccade tasks are performed continuously, the saccade landing point error data is obtained as follows: , , , , , which is fitted by this method , The same method was used to test the eye movement of Huntington's patients, and the error data of eye saccade landing point was obtained as follows: , , , , , fitting is obtained , .

[0069] In this embodiment, the error attenuation rate is obtained by fitting the landing point error, so that the long-term stability of the subject can be quantified. Traditionally, relying solely on the average landing point error cannot reflect the compensatory characteristics that change over time, while the landing point error attenuation constant can reveal latent functional decline and help improve the accuracy of early health monitoring.

[0070] In a possible embodiment, the method further includes: extracting frequency domain features of eye movements according to data of the saccade execution phase, specifically including the following steps: Step 1: According to the viewpoint position information corresponding to each sampling moment in the saccade execution stage, the instantaneous rate corresponding to each sampling moment is calculated to obtain the instantaneous rate sequence of the saccade execution stage.

[0071] In this step, the calculation method of the instantaneous rate sequence refers to the above-mentioned related embodiments.

[0072] Step 2: Performing Hilbert transform on the instantaneous rate sequence of the saccade execution phase to obtain an analytical signal corresponding to the instantaneous rate sequence.

[0073] In this step, the Hilbert transform is applied to generate the analytical signal s(t), whose mathematical expression is: ; Among them, the real part of the analytical signal is the instantaneous rate sequence of the saccade execution phase, the imaginary part is the result of Hilbert transform.

[0074] Step three: Determine the instantaneous amplitude envelope signal according to the analytical signal.

[0075] In this step, the envelope signal reflects the change of the velocity curve amplitude over time, and the instantaneous amplitude envelope signal is: .

[0076] Step 4: Calculate the power spectrum density of the instantaneous amplitude envelope signal.

[0077] In this step, the envelope signal can be Fourier transformed or the power spectrum density can be calculated using the Welch method, and then the dominant frequency component can be extracted. The corresponding formula is: ;in, The main frequency of the rate fluctuation reflects the main frequency characteristics of the change of eye saccade rate. is the instantaneous phase.

[0078] Step 5: Extracting the frequency domain features of eye movement according to the power spectrum density; the frequency domain features include dominant frequency components.

[0079] Step six: Generate the health reminder information based on the time domain static features, the time domain dynamic features and the frequency domain features.

[0080] In this step, the power spectrum density describes the distribution of signal power in the frequency domain. By calculating the power spectrum density, the distribution of signal power at different frequencies can be found, and then the dominant frequency component can be extracted, that is, the frequency component with a larger power share in the power spectrum density.

[0081] In one possible implementation, one or several peak points with the largest power value are found on the calculated power spectrum density curve. The frequencies corresponding to these peak points are the frequency components with a relatively large power ratio in the power spectrum density, that is, the candidate frequencies of the dominant frequency components. Certain screening criteria are set according to actual needs and signal characteristics. For example, the peak power is required to exceed a certain threshold (such as a certain proportion of the maximum power), or the frequency interval between adjacent peaks must meet certain conditions, etc., and the dominant frequency component is finally determined through screening.

[0082] For example, in saccade analysis, the dominant frequency component of the power spectral density of the saccade velocity curve of healthy people is concentrated in 8-12Hz, reflecting the normal cerebellar regulation rhythm; the dominant frequency of Alzheimer's disease patients is reduced to 3-5Hz, and abnormal high-frequency noise (14-18Hz) appears, indicating that the cortex-cerebellum loop is out of synchronization.

[0083] In this embodiment, by mapping the instantaneous rate sequence of the saccade execution phase to the frequency domain and extracting the frequency domain features, subtle abnormalities that cannot be identified in the time domain can be captured. The frequency domain features are combined with the time domain static features and the time domain dynamic features for multi-dimensional feature analysis, which can significantly improve the accuracy of early health risk monitoring in the nervous system function.

[0084] In this embodiment, the data obtained from the eye movement test is divided into three stages according to the occurrence of saccades in the time dimension: saccade latency stage, saccade execution stage and saccade stabilization stage (refer to Figure 4 ), then, the saccades in the execution phase of saccades are divided into main saccades and corrective saccades, and the time domain static features are extracted from each of them; the time domain dynamic features (the standard deviation of eye acceleration in the latent phase and the attenuation constant of the landing point error in the stable phase) are extracted based on the data of the saccade latency phase and the data of the saccade stability phase respectively; the frequency domain features are extracted based on the data of the saccade execution phase, and the comprehensive multi-scale and multi-dimensional features break through the limitations of traditional static feature analysis. Since the features of different dimensions can reflect different neural states of the brain, the effectiveness and comprehensiveness of the feature parameters are significantly improved from the time domain static dimension, dynamic compensation mechanism and frequency domain and other dimensions, which provides comprehensive and reliable data support for the health monitoring of early nervous system dysfunction.

[0085] In a possible embodiment, the saccade task includes one or more of a forward saccade task, a backward saccade task, and a memory saccade task.

[0086] Example, design reference for positive saccade task Figure 5As shown in the figure, at the beginning of each trial, a black cross fixation point FP appears in the center of the screen, and its size is 1 visual angle both horizontally and vertically. When FP appears, the subject looks at the fixation point as quickly as possible. If the subject's eyes enter the inspection window with a radius of 4 visual angles and FP as the center (the larger dotted circle in the figure) and stay in the window for 800 milliseconds (ms), FP disappears. At the same time, a solid stimulus point will randomly appear in one of the four peripheral positions of up, down, left, and right. The size of the stimulus point is 1 visual angle and 10 visual angles away from the center of the screen. After the stimulus point appears, the subject needs to look at the stimulus point as accurately and quickly as possible and keep fixating. At this time, if the subject's eyes enter the inspection window with a radius of 4 visual angles and stay for 300ms with the target stimulus point as the center, the stimulus point disappears and the trial ends. If the subject still does not look at FP after 1000 ms after the FP appears, or does not make an eye saccade after 2000 ms after the target stimulus point appears, the current trial ends. After each trial, there was an 800-ms inter-trial period during which no visual stimulus was displayed on the screen to avoid residual vision on the retina.

[0087] Design reference for the antisaccade task Figure 6 As shown in the figure, at the beginning of each trial, a black cross fixation point FP appears in the center of the screen, with a size of 1 visual angle horizontally × 1 visual angle vertically. When FP appears, the subject looks at the fixation point as quickly as possible. If the subject's eyes enter the inspection window with a radius of 4 visual angles and stay in the window for 800ms, FP disappears. At the same time, a solid stimulus point will randomly appear in one of the four peripheral positions of the upper, lower, left, and right. The size of the stimulus point is 1 visual angle and 10 visual angles away from the center of the screen. When the stimulus point appears, the subject cannot look at the stimulus point. On the contrary, the subject needs to look at the position of the opposite equidistant of the point as accurately and quickly as possible and keep fixating. At this time, if the subject's eyes enter the inspection window with a radius of 4 visual angles and stay for 300ms with the point opposite to the stimulus point as the center, the stimulus point disappears and the trial ends. If the subject still does not look at the FP after 1000 ms after the FP appears, or still does not make an eye saccade after 2000 ms after the stimulus point appears, the current trial ends. After each trial, there was an 800 ms inter-trial period during which no visual stimulus was displayed on the screen.

[0088] Design reference for memory saccade task Figure 7As shown in the figure, at the beginning of each trial, a fixation point FP will appear in the center of the screen, and the subject needs to look at the FP as quickly as possible. When the subject stares at the FP for the required time, the stimulus point will randomly appear in one of the four peripheral positions: up, down, left, and right. The subject needs to continue to keep his gaze on the FP point and remember the spatial position of the stimulus point. After 500ms, the stimulus point disappears, followed by a delay period of 1000ms. During the delay period, the subject still needs to keep his gaze on the FP. After the delay period, the FP point will disappear, and the subject needs to look at the position where the stimulus point appeared as accurately and quickly as possible.

[0089] Figure 8 is an application scenario diagram of a method for extracting multi-dimensional eye movement features provided by another embodiment of the present invention; Figure 8 As shown in FIG. 1 , the scenario includes an eye tracker, a test screen, and a cloud server. Another method of generating health prompt information will be described below in conjunction with this scenario.

[0090] In a possible embodiment, generating health prompt information based on the time-domain static features and the time-domain dynamic features includes: performing fully homomorphic encryption on the time-domain static features and the time-domain dynamic features based on (Ring Learning With Errors, RLWE) ciphertext to obtain encrypted time-domain static features and time-domain dynamic features; sending the encrypted time-domain static features and time-domain dynamic features to a cloud server, so that the cloud server compares the encrypted time-domain static features and time-domain dynamic features with a preset judgment threshold, and generates the health prompt information based on the comparison result.

[0091] It should be noted that fully homomorphic encryption is a privacy-enhancing technology that, in addition to protecting transmission confidentiality through traditional cryptographic algorithms, also supports arbitrary calculations on plaintext and ciphertext, and the result of the calculation is consistent with the result of direct calculation of the plaintext after decryption. Therefore, the eye movement features can be compared with the judgment threshold in an encrypted state without decryption, thereby protecting data privacy. In the present invention, the judgment threshold is stored in a cloud server, and the threshold comparison is implemented in the cloud, and the comparison algorithm in the fully homomorphic encryption scheme is used to achieve end-to-end privacy protection for early monitoring of nervous system functions.

[0092] In a possible implementation, assuming that the plain text value of the eye movement feature that needs privacy protection is a, and the plain text value of the judgment threshold that needs privacy protection is b, Taking the comparison judgment of as an example (the comparison algorithms of other features are similar), the input of the comparison algorithm is the RLWE ciphertext RLWE of encrypted a ( ), extract the constant term of RLWE ciphertext and convert it into LWE ciphertext. The algorithm output is LWE ciphertext LWE (1 / 0). If a and b satisfy If the comparison condition is met, the ciphertext output is 1, otherwise the ciphertext output is 0. Specifically, input RLWE( ) and then, by the formula Calculate the intermediate results , using the Extract algorithm from Extract LWE(out)=Extract(RLWE(out)). Among them, Represents the plaintext and ciphertext multiplication in the fully homomorphic encryption scheme, satisfying In polynomial ring operations, , by calculating ,when When , the constant term of the calculation result is 1 / 2, when When , the constant term is -1 / 2; Next, perform the homomorphic addition result and add the above result Perform homomorphic addition, when When , the polynomial constant term of the final plaintext is 1 / 2+1 / 2=1; when When , its constant term is 1 / 2-1 / 2=0. Finally, the Extract algorithm is executed, which is a sample extraction algorithm in the fully homomorphic encryption scheme. The input of the Extract algorithm is the RLWE ciphertext, and the output of the LWE ciphertext of its constant term is obtained to obtain the final LWE(1) or LWE(0).

[0093] For example, assuming N=4, a=1, b=2, according to the above calculation steps, the multiplication result is first obtained: , then its constant term is 1 / 2, the final result is 1 / 2+1 / 2=1, and the output is LWE(1).

[0094] In this embodiment, the terminal device adopts fully homomorphic encryption technology, and encrypts various eye movement features based on the RLWE ciphertext at the terminal to obtain the ciphertext form of the eye movement feature RLWE ( ), the ciphertext of the eye movement features is sent to the cloud server. The cloud server compares the eye movement features with the corresponding judgment threshold based on the ciphertext without decryption and generates health prompt information. On the cloud server, any attacker cannot obtain the ciphertext RLWE( ) to restore the plaintext value of the eye movement feature; the judgment threshold is stored in the cloud server, and the terminal device does not store the judgment threshold. The attacker cannot tamper with the threshold at will, achieving end-to-end privacy protection. In addition, the judgment threshold can be dynamically updated in the cloud server without relying on firmware upgrades.

[0095] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0096] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0097] Fig. 9 The structure diagram of the device for extracting multi-dimensional eye movement features provided by an embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows: like Fig. 9 As shown, the device for extracting multi-dimensional eye movement features includes: a data acquisition module 901, which is used to collect the viewpoint position information of the subject on the test screen according to a preset sampling frequency and record the sampling time corresponding to each viewpoint position information when the subject performs an eye movement test according to a predetermined eye movement task; a data processing module 902, which is used to determine the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; a feature extraction module 903, which is used to determine the time domain static features of eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and, determine the time domain dynamic features of eye movement according to the data of the saccade latency stage; an information generation module 904, which is used to generate health prompt information according to the time domain static features and the time domain dynamic features.

[0098] In a possible implementation, the data of the saccade latency stage include: saccade latency; the data of the saccade execution stage include: saccade amplitude, saccade duration and saccade rate; the data of the saccade stabilization stage include: saccade landing point error; the data processing module 902 is specifically used to: determine the start time and end time of the main saccade in each trial saccade task, as well as the start time and end time of each corrective saccade according to the viewpoint position information and the sampling time corresponding to each viewpoint position information; wherein the main saccade is the first saccade that occurs during the subject's scanning process from the fixation point to the target position, and the corrective saccade is the saccade after the end time of the main saccade; the difference between the time when the fixation point on the test screen disappears in each trial saccade task and the start time of the main saccade The absolute value of is used as the saccade latency of each trial saccade task; the saccade amplitude and saccade rate of each corrective saccade are determined according to the viewpoint position information corresponding to the start time of each corrective saccade and the viewpoint position information corresponding to the end time of each corrective saccade; the absolute value of the difference between the start time of each corrective saccade and the end time of each corrective saccade is used as the saccade duration of each corrective saccade; the saccade landing point error of each trial saccade task is determined according to the viewpoint position information corresponding to the target position on the test screen in each trial saccade task and the end time of the last corrective saccade; wherein the fixation point is the center point of the test screen, and the target position is the position of the stimulus point displayed on the test screen or the position of the symmetrical point of the stimulus point relative to the center point.

[0099] In a possible implementation, the feature extraction module 903 is specifically used to: calculate the average saccade latency corresponding to the n-trial saccade task according to the saccade latency of each saccade task; wherein n is an integer greater than zero; calculate the average saccade landing point error corresponding to the n-trial saccade task according to the saccade landing point error of each saccade task; calculate the average saccade amplitude of the corrective saccade in each saccade task according to the saccade amplitude of each corrective saccade; calculate the average saccade duration of the corrective saccade in each saccade task according to the saccade duration of each corrective saccade; calculate the saccade peak rate of the corrective saccade in each saccade task according to the saccade rate of each corrective saccade; and use the average saccade latency, the average saccade amplitude, the average saccade duration, the average saccade landing point error and the saccade peak rate as the time-domain static features.

[0100] In a possible implementation, the data of the saccadic latency stage also includes: the eye acceleration corresponding to each sampling moment in the saccadic latency stage; the data processing module 902 is also used to: calculate the instantaneous rate corresponding to each sampling time according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; determine the eye acceleration corresponding to each sampling time in the saccadic latency stage according to each sampling time and the instantaneous rate corresponding to each sampling time.

[0101] In a possible implementation, the feature extraction module 903 is further used to calculate the standard deviation of the eye acceleration within the saccade latency stage according to the eye acceleration corresponding to each sampling time within the saccade latency stage, and use the standard deviation of the eye acceleration as the time domain dynamic feature.

[0102] In a possible implementation, the data processing module 902 is also used to: calculate the instantaneous rate corresponding to each sampling moment according to the viewpoint position information corresponding to each sampling moment in the saccade execution stage, and obtain the instantaneous rate sequence of the saccade execution stage; perform Hilbert transform on the instantaneous rate sequence of the saccade execution stage to obtain the analytical signal corresponding to the instantaneous rate sequence; determine the instantaneous amplitude envelope signal according to the analytical signal; calculate the power spectral density of the instantaneous amplitude envelope signal; the feature extraction module 903 is also used to: extract the frequency domain characteristics of eye movement according to the power spectral density; the frequency domain characteristics include the dominant frequency component; the information generation module 904 is also used to generate the health prompt information according to the time domain static characteristics, the time domain dynamic characteristics and the frequency domain characteristics.

[0103] In a possible implementation, the device also includes: a data encryption module 905, which is used to perform fully homomorphic encryption on the time-domain static features and the time-domain dynamic features based on the RLWE ciphertext to obtain encrypted time-domain static features and time-domain dynamic features; send the encrypted time-domain static features and time-domain dynamic features to a cloud server, so that the cloud server compares the encrypted time-domain static features and time-domain dynamic features with a preset judgment threshold, and generates the health prompt information based on the comparison result.

[0104] Fig.10 Schematic diagram of an electronic device provided by an embodiment of the present invention. Fig.10 As shown, the electronic device 10 of this embodiment includes: a processor 100 and a memory 101. The memory 101 stores a computer program 102. When the processor 100 executes the computer program 102, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 100 executes the computer program 102, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0105] Exemplarily, the computer program 102 may be divided into one or more modules / units, which are stored in the memory 101 and executed by the processor 100 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 102 in the electronic device 10.

[0106] The electronic device 10 may include, but is not limited to, a processor 100 and a memory 101. Those skilled in the art will appreciate that Fig.10 It is only an example of the electronic device 10 and does not constitute a limitation of the electronic device 10. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 10 may also include input and output devices, network access devices, buses, etc.

[0107] The processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0108] The memory 101 may be an internal storage unit of the electronic device 10, such as a hard disk or memory of the electronic device 10. The memory 101 may also be an external storage device of the electronic device 10, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 10. Further, the memory 101 may also include both an internal storage unit of the electronic device 10 and an external storage device. The memory 101 is used to store the computer program 102 and other programs and data required by the electronic device 10. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0109] For the convenience and simplicity of description, only the division of the above functional modules / units is used as an example for illustration. In actual applications, the above functions can be assigned to different functional modules / units as needed. The above modules / units can be implemented in the form of hardware, software, or a combination of hardware and software.

[0110] The embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the methods in the above method embodiments are implemented.

[0111] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the methods in the above method embodiments are implemented.

[0112] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0113] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. If there is no special explanation or logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other. The technical features in different embodiments can be combined to form a new embodiment according to their internal logical relationship.

[0114] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for extracting multi-dimensional eye movement features, characterized in that: include: When the subject performs an eye movement test according to a predetermined eye movement task, the viewpoint position information of the subject on the test screen is collected according to a preset sampling frequency and the sampling time corresponding to each viewpoint position information is recorded; Determine data of the saccade latency stage, data of the saccade execution stage, and data of the saccade stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; Determining the time-domain static characteristics of eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and determining the temporal dynamic characteristics of eye movement according to the data of the saccade latency stage; Health reminder information is generated according to the time domain static features and the time domain dynamic features.

2. The method for extracting multi-dimensional eye movement features according to claim 1, characterized in that: The data of the saccade latency stage include: saccade latency; the data of the saccade execution stage include: saccade amplitude, saccade duration and saccade rate; the data of the saccade stability stage include: saccade landing point error; The method of determining data of the saccade latency stage, data of the saccade execution stage, and data of the saccade stabilization stage according to each viewpoint position information and the sampling time corresponding to each viewpoint position information comprises: According to the viewpoint position information and the sampling time corresponding to each viewpoint position information, determine the start time and end time of the main saccade in each trial saccade task, as well as the start time and end time of each corrective saccade; wherein the main saccade is the first saccade that occurs during the subject's scanning process from the fixation point to the target position, and the corrective saccade is the saccade that occurs after the end time of the main saccade; The absolute value of the difference between the time when the fixation point on the test screen disappears and the start time of the main saccade in each trial of the saccade task is used as the saccade latency of each trial of the saccade task; Determining the saccade amplitude and saccade rate of each corrective saccade according to the viewpoint position information corresponding to the start time of each corrective saccade and the viewpoint position information corresponding to the end time of each corrective saccade; The absolute value of the difference between the start time of each corrective saccade and the end time of each corrective saccade is used as the saccade duration of each corrective saccade; According to the viewpoint position information corresponding to the target position on the test screen and the end time of the last corrective saccade in each trial of the saccade task, the saccade landing point error of each trial of the saccade task is determined; The fixation point is the center point of the test screen, and the target position is the position of the stimulation point displayed on the test screen or the position of the symmetrical point of the stimulation point relative to the center point.

3. The method for extracting multi-dimensional eye movement features according to claim 2, characterized in that: Determining the time-domain static features of eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage includes: According to the saccade latency of each saccade task, calculating the average saccade latency corresponding to n saccade tasks; wherein n is an integer greater than zero; According to the saccade landing point error of each saccade task, calculating the average saccade landing point error corresponding to the n saccade tasks; Calculating the average saccade amplitude of the corrective saccades in each trial saccade task according to the saccade amplitude of each corrective saccade; Calculating the average saccade duration of the corrective saccades in each trial of the saccade task according to the saccade duration of each corrective saccade; Calculating the peak rate of the corrective saccades in each trial saccade task according to the saccade rate of each corrective saccade; The average saccade latency, the average saccade amplitude, the average saccade duration, the average saccade landing point error and the saccade peak rate are used as the time-domain static features.

4. The method for extracting multi-dimensional eye movement features according to claim 2, characterized in that: The data of the saccadic latency stage also includes: eyeball acceleration corresponding to each sampling moment in the saccadic latency stage; According to each viewpoint position information and the sampling time corresponding to each viewpoint position information, the data of the saccade latency stage is determined, and also includes: Calculate the instantaneous rate corresponding to each sampling time according to each viewpoint position information and the sampling time corresponding to each viewpoint position information; According to each sampling time and the instantaneous rate corresponding to each sampling time, the eyeball acceleration corresponding to each sampling time in the saccadic latency stage is determined.

5. The method for extracting multi-dimensional eye movement features according to claim 4, characterized in that: Determining the temporal dynamic characteristics of eye movement according to the data of the saccade latency stage includes: According to the eyeball acceleration corresponding to each sampling time in the eyeball latency stage, the standard deviation of the eyeball acceleration in the eyeball latency stage is calculated, and the standard deviation of the eyeball acceleration is used as the time domain dynamic feature.

6. The method for extracting multi-dimensional eye movement features according to claim 1, characterized in that: Also includes: According to the viewpoint position information corresponding to each sampling moment in the eye saccade execution stage, the instantaneous rate corresponding to each sampling moment is calculated to obtain the instantaneous rate sequence of the eye saccade execution stage; Performing Hilbert transform on the instantaneous rate sequence of the saccade execution phase to obtain an analytical signal corresponding to the instantaneous rate sequence; Determining an instantaneous amplitude envelope signal according to the analytical signal; Calculating the power spectral density of the instantaneous amplitude envelope signal; Extracting frequency domain features of eye movement according to the power spectrum density; the frequency domain features include dominant frequency components; The health prompt information is generated according to the time domain static features, the time domain dynamic features and the frequency domain features.

7. The method for extracting multi-dimensional eye movement features according to any one of claims 1 to 6, characterized in that: The saccade task includes one or more of a forward saccade task, a backward saccade task and a memory saccade task.

8. The method for extracting multi-dimensional eye movement features according to any one of claims 1 to 6, characterized in that: The generating of health prompt information according to the time domain static features and the time domain dynamic features includes: Performing fully homomorphic encryption on the time-domain static features and the time-domain dynamic features based on the RLWE ciphertext to obtain encrypted time-domain static features and time-domain dynamic features; The encrypted time-domain static features and time-domain dynamic features are sent to a cloud server, so that the cloud server compares the encrypted time-domain static features and time-domain dynamic features with a preset judgment threshold, and generates the health prompt information based on the comparison result.

9. A device for extracting multi-dimensional eye movement features, characterized in that: include: The data acquisition module is used to collect the viewpoint position information of the subject on the test screen according to the preset sampling frequency and record the sampling time corresponding to each viewpoint position information when the subject performs the eye movement test according to the predetermined eye movement task. A data processing module, used for determining data of a saccade latency stage, data of a saccade execution stage and data of a saccade stabilization stage according to each viewpoint position information and a sampling time corresponding to each viewpoint position information; A feature extraction module, used for determining the time-domain static features of eye movement according to the data of the saccade latency stage, the data of the saccade execution stage and the data of the saccade stabilization stage; and determining the temporal dynamic characteristics of eye movement according to the data of the saccade latency stage; The information generation module is used to generate health reminder information according to the time domain static characteristics and the time domain dynamic characteristics.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images