Eye jump feature extraction method and device and electronic equipment

Abstract

The invention is suitable for the technical field of electric digital data processing, and provides an eye jump feature extraction method and device and electronic equipment, and the method comprises the steps: obtaining the eye movement track data of a target tested person collected in each test time in an eye jump test process; for the eye movement trajectory data of each trial, determining a corresponding eye movement speed sequence, and determining a plurality of eye jump data segments from the eye movement trajectory data according to the eye movement speed sequence; determining the latency period and the amplitude of each eye jump data segment, and screening a main eye jump data segment from the plurality of eye jump data segments according to the latency period and the amplitude; and extracting eye jump features based on the main eye jump data segments of all trials. According to the method, all eye jump characteristics can be accurately extracted from the eye movement data, and a basis is provided for accurate detection of cognitive impairment.

Description

Technical Field

[0001] The invention belongs to the technical field of electronic digital data processing, and in particular relates to a method and device for extracting eye saccade features, and electronic equipment. Background Art

[0002] Patients with degenerative diseases in the elderly (such as Alzheimer's disease and Parkinson's disease) usually show cognitive impairment. Cognitive impairment refers to the impairment of one or more cognitive functions, which leads to difficulties in learning, memory, thinking, communication and daily life, including memory impairment, rehearsal impairment, attention impairment, thinking impairment, visual-spatial impairment, etc.

[0003] In recent years, eye movement testing technology has gradually been applied to the early detection of cognitive impairment. People with different types and degrees of cognitive impairment may have different eye movement characteristics. Through eye movement characteristics, we can deeply understand the cognitive impairment pattern of each cognitive impairment person and provide a basis for the formulation of personalized rehabilitation programs. For example, for cognitive impairment people with attention deficits, their eye movement characteristics may be manifested as frequent shifts of gaze points and difficulty focusing on target objects; for cognitive impairment people with memory impairments (such as Alzheimer's disease patients), when performing recall-related tasks or facing familiar scenes, their eye movement characteristics may be manifested as purposeless and disordered saccades; for cognitive impairment people with executive dysfunction (such as Parkinson's disease patients), in visual tasks that require planning, organization and decision-making, their eye movement characteristics may be manifested as the inability of eye movements to effectively switch and evaluate between different targets in an orderly manner. Eye movement testing can reveal potential problems in patients' cognition, emotions and neurological functions by tracking physiological parameters such as eye movement trajectory, gaze point, and movement speed. Among them, saccades are a rapid eye movement whose purpose is to quickly shift the line of sight from one visual target to another. Although this movement seems simple, it is actually coordinated and controlled by multiple areas of the brain, including the frontal lobe eye movement area, parietal lobe, basal ganglia, thalamus, brainstem and other structures. Therefore, eye saccades are not only a movement process, but also an external manifestation of higher-order brain functions, involving cognitive processes such as attention, inhibitory control, working memory, and decision-making. Therefore, eye saccades can be used as a behavioral probe to indirectly reflect specific brain functions.

[0004] At present, the main challenge of eye movement feature extraction is how to accurately extract effective features from eye movement data under the interference of various external factors. Cognitive impairment test systems in related technologies usually establish eye movement models based on the physiological mechanism and movement characteristics of eye movements, such as dynamic models based on physical principles or neural control models based on physiology, and extract eye movement features by simply fitting eye movement data into the model. However, in the actual eye movement test process, since the subjects are easily disturbed by various factors such as lighting and eye health, resulting in multiple eye saccades, existing eye movement models are difficult to fully and accurately extract these eye movement features, thereby greatly reducing the accuracy of cognitive impairment detection. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide a method, device and electronic device for extracting eye saccade features to accurately extract all eye saccade features from eye movement data, thereby providing a basis for accurate detection of cognitive impairment.

[0006] A first aspect of an embodiment of the present invention provides a method for extracting eye saccade features, comprising: Obtain the eye movement trajectory data of the target subject collected in each trial during the eye saccade test; For each trial of eye movement trajectory data, determining a corresponding eye movement velocity sequence, and determining a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence; Determine the latency and amplitude of each saccade data segment, and select the main saccade data segment from the plurality of saccade data segments according to the latency and the amplitude; Based on the main saccade data segments of all trials, saccade features are extracted.

[0007] In a possible implementation, determining a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence includes: According to a preset speed threshold and the eye movement speed sequence, the eye movement trajectory data is divided into a plurality of eye saccade data segments and a plurality of non-eye saccade data segments; Determine an average eye movement velocity of a plurality of non-saccade data segments, and determine a new velocity threshold according to the average eye movement velocity of the plurality of non-saccade data segments; The new speed threshold is used as the preset speed threshold, and the step of dividing the eye movement trajectory data into multiple saccade data segments and multiple non-saccade data segments according to the preset speed threshold and the eye movement speed sequence is re-executed until the saccade data segment no longer changes, and the saccade data segment that no longer changes is determined as the final saccade data segment.

[0008] In a possible implementation, dividing the eye movement trajectory data into a plurality of eye saccade data segments and a plurality of non-eye saccade data segments according to a preset speed threshold and the eye movement speed sequence includes: Determine the eye movement data as an eye saccade data segment, in which the eye movement speed is greater than the speed threshold and exceeds a preset number threshold; The continuous data points except the eye-saccade data segment are determined as non-eye-saccade data segments.

[0009] In a possible implementation, determining a new speed threshold according to an average eye movement speed of a plurality of non-saccade data segments includes: Determine the standard deviation based on the average eye movement speed of multiple non-saccade data segments; The sum of the average eye movement velocity and the preset multiple of the standard deviation is used as the new velocity threshold.

[0010] In a possible implementation, the eye saccade test process includes: When the target subject looks at the fixation point, displaying a target point at a position different from the fixation point, so that the target subject looks at the target point and generates eye saccades; Determining the latency and amplitude of each saccade data segment includes: Determine the latency period according to the difference between the start time of each saccade data segment and the display time of the target point; The amplitude is determined according to the viewing angle at the start and end of each saccade data segment.

[0011] In a possible implementation, the selecting a main saccade data segment from the plurality of saccade data segments according to the latency and the amplitude includes: Determine the saccade data segment whose latency is a positive number and whose amplitude is greater than a preset amplitude threshold as the main saccade data segment; Wherein, a positive number of the latency period indicates that the saccade occurs after the target point appears.

[0012] In a possible implementation, before determining the corresponding eye movement velocity sequence for the eye movement trajectory data of each trial, the method further includes: Set sliding windows of preset size and step length; The eye movement trajectory data in the sliding window are fitted with a second-order polynomial and the median of the fitting curve is taken to perform moving median filtering on the eye movement trajectory data.

[0013] In a possible implementation, extracting saccade features based on the main saccade data segments of all trials includes: Determining target saccade features according to the type of cognitive impairment to be detected, and extracting the target saccade features based on the main saccade data segments of all trials; The target eye saccade feature includes at least one of the following: The amplitude average of the main saccade data segment for all trials; The average end time of the main saccade data segment for all trials; The farthest distance between each data point of each main saccade data segment and the fixation point; The angle error between the last data point of each main saccade data segment and the fixation point; the duration of each major saccade data segment; The total duration of all major saccade segments for each trial.

[0014] A second aspect of an embodiment of the present invention provides a device for extracting eye saccade features, comprising: An acquisition module is used to acquire the eye movement trajectory data of the target subject collected in each trial during the eye saccade test; A processing module, for determining a corresponding eye movement velocity sequence for each trial of eye movement trajectory data, and determining a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence; A screening module, used for determining a latency and an amplitude of each saccade data segment, and screening a main saccade data segment from the plurality of saccade data segments according to the latency and the amplitude; The extraction module is used to extract eye saccade features based on the main eye saccade data segments of all trials.

[0015] A third aspect of an embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements the steps in the first aspect or any possible implementation method of the first aspect.

[0016] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in the above-mentioned first aspect or any possible implementation method of the first aspect.

[0017] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: After collecting the eye movement trajectory data, the embodiment of the present invention determines multiple saccade data segments from the eye movement trajectory data through the eye movement velocity sequence corresponding to the eye movement trajectory data, that is, the preliminary detection of saccades is achieved based on the eye movement velocity; further, the saccade data segments are screened through the latency and amplitude of each saccade data segment to obtain the main saccade data segments, and the data that may be misjudged as saccades is removed, thereby avoiding the influence of those fast but non-saccade data; finally, the saccade features can be extracted through the main saccade data segments of all trials. The embodiment of the present invention can accurately extract all saccade features from the eye movement data, providing a basis for the accurate detection of cognitive impairment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 This is an example of an application scenario of the eye saccade feature extraction method provided by an embodiment of the present invention. Figure 1 ; Figure 2 This is an example of an application scenario of the eye saccade feature extraction method provided by an embodiment of the present invention. Figure 2 ; Figure 3 Schematic diagram of the implementation process of the eye saccade feature extraction method provided by an embodiment of the present invention; Figure 4 is a schematic diagram of a device for extracting eye saccade features provided by an embodiment of the present invention; Figure 5 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0021] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0022] Saccades are a typical eye movement feature, which aims to quickly shift the gaze from one visual target to another. Saccadic test is the main way to obtain saccadic data, which can include three saccadic task tests: forward, reverse and memory. Forward saccadic task is used to test reflexive saccades, and reverse and memory saccade tasks are used to test autonomous saccades.

[0023] For example, the eye saccade test scenario can be found in Figure 1 shown.

[0024] The subject wears an eye tracker and quickly looks at the fixation point in the center of the test screen at the beginning of the eye movement test. Then, when the fixation point disappears, a target point appears on the test screen. The subject needs to quickly look at the target point as required. In this process, the eye tracker collects the subject's eye image according to a certain sampling frequency domain, and then processes the eye image to obtain the subject's viewpoint position on the test screen. The viewpoint positions corresponding to all sampling moments constitute an eye movement trajectory during the eye movement test. The eye tracker uploads the collected data to the cloud server for subsequent data analysis.

[0025] Here, according to the driving factors of saccades, saccades can be divided into reflexive saccades and autonomous saccades, and corresponding test tasks can be designed according to the driving factors of saccades, such as forward saccade tasks, reverse saccade tasks, and memory saccade tasks.

[0026] The design of the forward saccade task can be found in Figure 2 As shown in the figure, the middle cross represents the initial fixation point of the eye, the dot represents the target point, the circle represents the inspection area, and the arrow represents the direction of the subject's eye saccade. After the experiment begins, a cross fixation point appears in the center of the screen, with a size of 1 visual angle in the horizontal direction and 1 visual angle in the vertical direction (the angle between the eye and the observed target). The subject needs to stare at the cross. After 800 ms, the central cross fixation point disappears and the target point appears. The subject needs to look at the target point quickly. Once the subject's eyes enter the inspection area with a radius of 4 visual angles centered on the target point and stay there for 300 ms, it is considered that the task of a trial is successfully completed. If the subject's eyes fail to enter the inspection area within 1000 ms after the appearance of the cross fixation point, or if no eye saccade occurs within 2000 ms after the appearance of the target point, the trial will be considered a failure. After each trial, in order to avoid visual residue on the retina, the screen will present a blank screen for 800 ms to separate each trial.

[0027] Similarly, in the reverse saccade task, after the target point appears, the subjects need to quickly look at the opposite position of the target point. In the memory saccade task, after the target point appears, the subjects continue to stare at the cross fixation point and remember the position of the target point. The target point will disappear after 500ms, followed by a delay period of 1000ms. During the delay period, the subjects still need to keep their gaze on the cross fixation point. After the delay period, the cross fixation point will disappear, and the subjects need to quickly look at the position where the target point appeared. Among them, the settings of each time in the three saccade task tests of forward, reverse and memory can be flexibly adjusted according to actual needs to meet the testing needs in different scenarios.

[0028] The specific eye saccade task test is not limited in this embodiment. Three eye saccade task tests can be performed in sequence.

[0029] It should be noted that most of the traditional studies on saccade-related parameters are aimed at the primary saccade, that is, the first saccade that occurs during the scan from the fixation point to the target point. Under normal circumstances, saccades are fast and accurate eye movements that help us shift our sight to the target point. If the first saccade does not reach the target point accurately, a corrective saccade will occur. When the subjects are undergoing saccade task tests, they are easily disturbed by various factors such as environmental factors and eye health factors, resulting in abnormal primary saccades, followed by multiple corrective saccades. In this case, the eye movement model in the related technology only analyzes the relevant parameters of the primary saccade alone, which is difficult to fully reflect the true state of the subjects.

[0030] To solve the above problems, this embodiment designs a method for extracting eye saccade features. Figure 3 As shown, including: Step S301, obtaining the eye movement trajectory data of the target subject collected in each trial during the eye saccade test.

[0031] In this embodiment, a high-precision rapid eye tracker can be used to track both eyes in real time at a sampling frame rate of 1000 times / second to obtain eye movement trajectory data. The eye movement trajectory data here can be in the form of the position coordinates of the line of sight on the test screen. Further, in order to facilitate analysis, the eye movement trajectory data can be converted into the form of viewing angles. For example, for a screen resolution of 1920×1080 For a screen with a viewing angle of 100,000, the conversion formula is as follows:

[0032]

[0033] in, is the horizontal coordinate of the screen, y The vertical coordinate of the screen.

[0034] Here, this embodiment unifies the measurement standard by converting the eye movement trajectory data into the form of viewing angle, eliminates the interference caused by differences in individual eye physiological structures, and more accurately describes the changes in the position and direction of eye movements in space in the form of viewing angle.

[0035] Step S302: for the eye movement trajectory data of each trial, determine the corresponding eye movement velocity sequence, and determine a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence.

[0036] The eye movement velocity refers to the movement speed of the eyes at different times. In this embodiment, the eye movement velocity in each time interval can be obtained by calculating the change in the eye viewing angle between adjacent time points and dividing it by the time interval. The eye movement velocities calculated at different time points are arranged in chronological order to form an eye movement velocity sequence.

[0037] Saccades are when the subject moves their gaze from the fixation point to the target point. In the eye movement velocity sequence, saccades are usually manifested as a stage in which the speed suddenly increases and the amplitude is large. According to the characteristics of the eye movement velocity sequence (for example, a speed threshold is set, and when the eye movement velocity exceeds this threshold, it is considered that a saccade may have occurred), the time period and position information corresponding to the occurrence of saccades are found from the eye movement trajectory data, and the eye movement trajectory data in these time periods constitute saccade data segments. In other words, the eye movement velocity sequence is used as the basis for judgment, and the part of the eye movement trajectory data that belongs to saccades is extracted to form multiple independent saccade data segments, so that the specific eye movement behavior of saccades can be analyzed and studied later.

[0038] Step S303, determining the latency and amplitude of each saccade data segment, and selecting a main saccade data segment from the plurality of saccade data segments according to the latency and amplitude.

[0039] Here, the incubation period The definition of is the difference between the start time of each saccade data segment and the display time of the target point:

[0040] in, For the i The onset time of the saccade, The target point appearance time.

[0041] In the test, the subjects' eye saccade behavior is a reaction to the appearance of the target point, so under normal circumstances, eye saccades should occur after the target point appears, so the latency must be a positive number. If the latency of a certain eye saccade data segment is a negative number, it means that the eye saccade has already started before the target point appears, which is not in line with normal expectations, so these eye saccade data segments with negative latency are deleted to ensure the rationality and validity of the data.

[0042] amplitude A The definition of is the difference in visual angle between the start and end of each saccade data segment:

[0043] in, is the horizontal viewing angle at the end, is the horizontal viewing angle at the start, is the vertical viewing angle at the end, is the starting vertical viewing angle.

[0044] The amplitude reflects the magnitude of the change in eye angle during saccade. Saccade data segments with amplitudes less than the amplitude threshold (this threshold is determined according to the specific requirements of the experiment and the characteristics of the data, and is used to distinguish large and significant saccades, such as 2 visual angles) can be deleted to avoid the influence of fast but non-saccade data.

[0045] Step S304: extracting saccade features based on the main saccade data segments of all trials.

[0046] Cognitive impairment covers many different types, and different types of cognitive impairment differ in physiological mechanisms, manifestations, and effects on brain function. For example, some types may show abnormalities in the accuracy, speed, and latency of saccades, and may have more erroneous saccades, or slower saccade speed and longer latency. In addition to problems with saccade speed and accuracy, some types may also have difficulties in initiating saccades, which manifests as prolonged saccade reaction time.

[0047] In this embodiment, the key saccade features corresponding to different types of cognitive impairment can be identified and extracted through experiments or correlation analysis.

[0048] Exemplarily, eye saccade features may include but are not limited to: (1) The average amplitude of the main saccade data segment of all trials :

[0049] in, For the iThe amplitude of the main eye movement data segment.

[0050] (2) Average end time of the main saccade data segment of all trials :

[0051] in, is the end time of the main saccade data segment.

[0052] (3) The maximum distance between each data point and the fixation point in each main saccade data segment :

[0053]

[0054] in, For the main eye saccade data segment i The horizontal viewing angle of the data point, For the main eye saccade data segment i The vertical viewing angle of the data point, 、 is the viewing angle of the gaze point.

[0055] (4) The angle error between the last data point of each main saccade data segment and the fixation point :

[0056] in, and is the last data point of the main saccade data segment Direction viewing angle and Direction viewing angle, To convert radians to degrees.

[0057] (5) Duration of each major saccade data segment :

[0058] in, is the time corresponding to the end point of the main eye saccade data segment, The time corresponding to the starting point of the main eye saccade data segment.

[0059] (6) The total duration of all main saccade data segments for each trial :

[0060]

[0061] in, The end time of the trial. is the latency of the first major saccade data segment.

[0062] For example, for cognitively impaired people with executive dysfunction, in visual tasks that require planning, organization, and decision-making, their eye movement characteristics may be manifested as the inability to effectively and orderly switch between different targets. In this embodiment, after extracting the above-mentioned eye saccade features, the end time average of the main eye saccade data segments of all trials can be used to calculate the average end time of the main eye saccade data segments of all trials. , the angle error between the last data point of each main saccade data segment and the fixation point , and compared with the corresponding thresholds to determine whether the target subject has cognitive impairment of attention deficit.

[0063] After collecting the eye movement trajectory data, the embodiment of the present invention determines multiple saccade data segments from the eye movement trajectory data through the eye movement velocity sequence corresponding to the eye movement trajectory data, that is, the preliminary detection of saccades is achieved based on the eye movement velocity; further, the saccade data segments are screened through the latency and amplitude of each saccade data segment to obtain the main saccade data segments, and the data that may be misjudged as saccades is removed, thereby avoiding the influence of those fast but non-saccade data; finally, the saccade features can be extracted through the main saccade data segments of all trials. The embodiment of the present invention can accurately extract all saccade features from the eye movement data, providing a basis for the accurate detection of cognitive impairment.

[0064] In some embodiments, after acquiring the eye movement track data collected in each trial in step S301, an improved moving median filtering method may be used to filter the eye movement track data.

[0065] Exemplarily, a sliding window with a preset size (e.g., 5) and step size (e.g., 1) can be set. In each sliding window containing 5 data points, the least squares method is used to fit the data in the window with a second-order polynomial, and the median of the fitting curve is used as the smoothed result, so as to more accurately retain the trend information of the eye movement data and effectively suppress the influence of instantaneous outliers. Subsequently, eye movement feature extraction is performed based on the filtered data to further improve the accuracy of the extraction result. Optionally, the filtering formula is:

[0066] in, is the time index within the window, 、 、 are the fitting coefficients.

[0067] In some embodiments, for step S302, a method for detecting eye saccades with an adaptive speed threshold is designed to improve detection accuracy. The steps include: Step 1: According to a preset speed threshold and an eye movement speed sequence, the eye movement trajectory data is divided into a plurality of saccade data segments and a plurality of non-saccade data segments.

[0068] Assume that the velocity sequence of the eye movement trajectory is ,in Indicates The speed of each data point. The possible starting and ending points of saccades can be found with a speed threshold of 15 (viewing angles / second) (points that are continuous for 20ms or more are considered saccades), and the saccade data segment and the non-saccade data segment can be divided. The speed threshold here is determined based on experiments and experience.

[0069] Step 2: determine the average eye movement speed of multiple non-saccade data segments, and determine a new speed threshold according to the average eye movement speed of multiple non-saccade data segments.

[0070] Exemplarily, the standard deviation may be determined based on the average eye movement speed of multiple non-saccade data segments, and the sum of the average eye movement speed and a preset multiple of the standard deviation may be used as a new speed threshold.

[0071] Optionally, the average speed of the non-saccadic segments Plus 2.58 times the standard deviation As the new speed threshold:

[0072] 2.58 times the standard deviation is chosen because it can help determine a relatively reasonable threshold value for distinguishing between normal speed ranges and abnormal speed ranges. For example, in normally distributed data, approximately 99% of the data will fall within the range of the mean plus or minus 2.58 times the standard deviation. Therefore, when the average speed plus 2.58 times the standard deviation of the average speed is used as the new speed threshold, speeds that exceed this threshold can be considered abnormal for further analysis or appropriate measures.

[0073] Step three: according to the new speed threshold, the eye movement trajectory data is re-divided into a plurality of saccade data segments and a plurality of non-saccade data segments.

[0074] Step 4: repeat step 2 and step 3 until the saccade data segment no longer changes, and determine the saccade data segment that no longer changes as the final saccade data segment.

[0075] Finally, the continuous data points in the eye movement trajectory data whose eye movement speed is greater than the speed threshold and exceeds the preset number threshold are determined as eye saccade data segments, and the continuous data points other than the eye saccade data segments are determined as non-eye saccade data segments. For example, the data points whose eye movement speed is greater than the speed threshold and is continuous for 20ms or more are found as eye saccade data segments, and the first data point is the starting point of the eye saccade, and the last data point is the end point of the eye saccade.

[0076] Compared with a fixed speed threshold, this embodiment can detect eye saccade data segments more accurately.

[0077] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0078] Figure 4 is a schematic diagram of an eye saccade feature extraction device provided by an embodiment of the present invention, see Figure 4 As shown, the eye saccade feature extraction device 40 includes: The acquisition module 41 is used to acquire the eye movement trajectory data of the target subject collected in each trial during the eye saccade test.

[0079] The processing module 42 is used to determine the corresponding eye movement velocity sequence for the eye movement trajectory data of each trial, and determine a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence.

[0080] The screening module 43 is used to determine the latency and amplitude of each saccade data segment, and screen the main saccade data segment from the multiple saccade data segments according to the latency and amplitude.

[0081] The extraction module 44 is used to extract eye saccade features based on the main eye saccade data segments of all trials.

[0082] In a possible implementation, the processing module 42 is used to: According to a preset speed threshold and an eye movement speed sequence, the eye movement trajectory data is divided into a plurality of eye saccade data segments and a plurality of non-eye saccade data segments; Determine an average eye movement velocity of a plurality of non-saccade data segments, and determine a new velocity threshold according to the average eye movement velocity of the plurality of non-saccade data segments; The new speed threshold is used as the preset speed threshold, and the step of dividing the eye movement trajectory data into multiple saccade data segments and multiple non-saccade data segments according to the preset speed threshold and the eye movement speed sequence is re-executed until the saccade data segment no longer changes, and the saccade data segment that no longer changes is determined as the final saccade data segment.

[0083] In a possible implementation, the processing module 42 is used to: Determine the eye movement data segments as continuous data points in which the eye movement speed is greater than the speed threshold and exceeds the preset number threshold; The continuous data points except the saccade data segment are determined as the non-saccade data segment.

[0084] In a possible implementation, the processing module 42 is used to: Determine the standard deviation based on the average eye movement speed of multiple non-saccade data segments; The sum of the average eye movement velocity and the preset multiple standard deviation is taken as the new velocity threshold.

[0085] In a possible implementation, the acquisition module 41 is used to: When the target subject looks at the fixation point, a target point different from the fixation point is displayed to make the target subject look at the target point and produce eye saccades; The screening module 43 is used to: Determine the latency period according to the difference between the start time of each saccade data segment and the display time of the target point; The amplitude was determined based on the viewing angle at the start and end of each saccade data segment.

[0086] In a possible implementation, the screening module 43 is used to: Determine the saccade data segment whose latency is a positive number and whose amplitude is greater than a preset amplitude threshold as the main saccade data segment; A positive latency indicates that the saccade occurred after the target point appeared.

[0087] In a possible implementation, after acquiring the eye movement trajectory data collected in each trial, the acquisition module 41 is further used to: Set sliding windows of preset size and step length; The eye movement trajectory data in the sliding window are fitted with a second-order polynomial and the median of the fitting curve is taken to perform moving median filtering on the eye movement trajectory data.

[0088] In a possible implementation, the extraction module 44 is used to: Determine the target saccade features according to the type of cognitive impairment to be detected, and extract the target saccade features based on the main saccade data segments of all trials; The target eye saccade feature includes at least one of the following: The amplitude average of the main saccade data segment for all trials; The average end time of the main saccade data segment for all trials; The maximum distance between each data point and the fixation point in each main saccade data segment; The angle error between the last data point of each main saccade data segment and the fixation point; the duration of each major saccade data segment; The total duration of all major saccade segments for each trial.

[0089] After collecting the eye movement trajectory data, the embodiment of the present invention determines multiple saccade data segments from the eye movement trajectory data through the eye movement velocity sequence corresponding to the eye movement trajectory data, that is, the preliminary detection of saccades is achieved based on the eye movement velocity; further, the saccade data segments are screened through the latency and amplitude of each saccade data segment to remove data that may be misjudged as saccades, thereby avoiding the influence of those fast but non-saccade data; finally, the saccade features can be extracted through the main saccade data segments of all trials. The embodiment of the present invention can accurately extract all saccade features from the eye movement data, providing a basis for the accurate detection of cognitive impairment.

[0090] Figure 5 FIG. 5 is a schematic diagram of an electronic device 50 provided by an embodiment of the present invention. Figure 5 As shown, the electronic device 50 of this embodiment includes: a processor 51, a memory 52, and a computer program 53 stored in the memory 52 and executable on the processor 51, such as an eye saccade feature extraction program. When the processor 51 executes the computer program 53, the steps in the above-mentioned eye saccade feature extraction method embodiments are implemented, such as Figure 3 Alternatively, when the processor 51 executes the computer program 53, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 4 The functions of the modules 41 to 44 are shown.

[0091] Exemplarily, the computer program 53 may be divided into one or more modules / units, which are stored in the memory 52 and executed by the processor 51 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 53 in the electronic device 50.

[0092] The electronic device 50 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 50 may include, but is not limited to, a processor 51 and a memory 52. ​​Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 50 and does not constitute a limitation of the electronic device 50. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 50 may also include input and output devices, network access devices, buses, etc.

[0093] The processor 51 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0094] The memory 52 may be an internal storage unit of the electronic device 50, such as a hard disk or memory of the electronic device 50. The memory 52 may also be an external storage device of the electronic device 50, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 50. Further, the memory 52 may also include both an internal storage unit of the electronic device 50 and an external storage device. The memory 52 is used to store the computer program and other programs and data required by the electronic device 50. The memory 52 may also be used to temporarily store data that has been output or is to be output.

[0095] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0096] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0097] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0098] In the embodiments provided by the present invention, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0099] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0101] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0102] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for extracting eye saccade features, characterized in that: include: Obtain the eye movement trajectory data of the target subject collected in each trial during the eye saccade test; For each trial of eye movement trajectory data, determining a corresponding eye movement velocity sequence, and determining a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence; Determine the latency and amplitude of each saccade data segment, and select the main saccade data segment from the plurality of saccade data segments according to the latency and the amplitude; Based on the main saccade data segments of all trials, saccade features are extracted.

2. The eye saccade feature extraction method according to claim 1, wherein: Determining a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence includes: According to a preset speed threshold and the eye movement speed sequence, the eye movement trajectory data is divided into a plurality of eye saccade data segments and a plurality of non-eye saccade data segments; Determine an average eye movement velocity of a plurality of non-saccade data segments, and determine a new velocity threshold according to the average eye movement velocity of the plurality of non-saccade data segments; The new speed threshold is used as the preset speed threshold, and the step of dividing the eye movement trajectory data into multiple saccade data segments and multiple non-saccade data segments according to the preset speed threshold and the eye movement speed sequence is re-executed until the saccade data segment no longer changes, and the saccade data segment that no longer changes is determined as the final saccade data segment.

3. The eye saccade feature extraction method as claimed in claim 2, characterized in that: The step of dividing the eye movement trajectory data into a plurality of eye saccade data segments and a plurality of non-eye saccade data segments according to a preset speed threshold and the eye movement speed sequence comprises: Determine the eye movement data as an eye saccade data segment, in which the eye movement speed is greater than the speed threshold and exceeds a preset number threshold; The continuous data points except the eye-saccade data segment are determined as non-eye-saccade data segments.

4. The eye saccade feature extraction method as claimed in claim 2, characterized in that: Determining a new speed threshold according to the average eye movement speed of multiple non-saccade data segments includes: Determine the standard deviation based on the average eye movement speed of multiple non-saccade data segments; The sum of the average eye movement velocity and the preset multiple of the standard deviation is used as the new velocity threshold.

5. The eye saccade feature extraction method according to any one of claims 1 to 4, characterized in that: The eye saccade test process includes: When the target subject looks at the fixation point, displaying a target point at a position different from the fixation point, so that the target subject looks at the target point and generates eye saccades; Determining the latency and amplitude of each saccade data segment includes: Determine the latency period according to the difference between the start time of each saccade data segment and the display time of the target point; The amplitude is determined according to the viewing angle at the start and end of each saccade data segment.

6. The eye saccade feature extraction method as claimed in claim 5, characterized in that: The step of selecting a main eye saccade data segment from the plurality of eye saccade data segments according to the latency and the amplitude comprises: Determine the saccade data segment whose latency is a positive number and whose amplitude is greater than a preset amplitude threshold as the main saccade data segment; Wherein, a positive number of the latency period indicates that the saccade occurs after the target point appears.

7. The eye saccade feature extraction method according to any one of claims 1 to 4, characterized in that: Before determining the corresponding eye movement velocity sequence for the eye movement trajectory data of each trial, it also includes: Set sliding windows of preset size and step length; The eye movement trajectory data in the sliding window are fitted with a second-order polynomial and the median of the fitting curve is taken to perform moving median filtering on the eye movement trajectory data.

8. The eye saccade feature extraction method as claimed in claim 5, characterized in that: Extracting eye saccade features based on the main eye saccade data segments of all trials includes: Determining target saccade features according to the type of cognitive impairment to be detected, and extracting the target saccade features based on the main saccade data segments of all trials; The target eye saccade feature includes at least one of the following: The amplitude average of the main saccade data segment for all trials; The average end time of the main saccade data segment for all trials; The farthest distance between each data point of each main saccade data segment and the fixation point; The angle error between the last data point of each main saccade data segment and the fixation point; the duration of each major saccade data segment; The total duration of all major saccade segments for each trial.

9. A device for extracting eye saccade features, characterized in that: include: An acquisition module is used to acquire the eye movement trajectory data of the target subject collected in each trial during the eye saccade test; A processing module, for determining a corresponding eye movement velocity sequence for each trial of the eye movement trajectory data, and determining a plurality of eye saccade data segments from the eye movement trajectory data according to the eye movement velocity sequence; A screening module, used for determining a latency and an amplitude of each saccade data segment, and screening a main saccade data segment from the plurality of saccade data segments according to the latency and the amplitude; The extraction module is used to extract eye saccade features based on the main eye saccade data segments of all trials.

10. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images