A pupil-based cognitive abnormality detection method and device and a medium

Abstract

This specification discloses a pupil-based cognitive anomaly detection method, device, and medium, relating to the field of artificial intelligence technology, to address the problems of low efficiency and poor accuracy in current cognitive anomaly detection. The method includes: determining the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; transmitting the detection task content to the display terminal for display, and receiving eye-tracking flow data collected by an eye tracker during the display process; calling the eye-tracking flow data to be analyzed corresponding to each detection task type, and extracting the features to be detected from the corresponding eye-tracking flow data based on the detection strategy corresponding to each detection task type; and inputting the features to be detected into a pre-trained discriminant model of the corresponding type to obtain the cognitive anomaly detection result for the current user.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a method, device and medium for detecting cognitive anomalies based on pupils. Background Technology

[0002] With the increasing specialization and complexity of occupations in modern society, high-intensity and high-risk work scenarios are becoming more common. In occupational settings such as aerospace, rail transportation, and chemical production, core cognitive abilities such as attention and reaction speed have become key factors affecting occupational safety and work efficiency, and even subtle abnormalities in cognitive function can lead to serious consequences. Therefore, for these positions, it is crucial to conduct rapid screening of core cognitive abilities such as attention and reaction speed before employment to detect cognitive abnormalities.

[0003] Traditional cognitive assessments rely on manual methods such as questionnaires or question-and-answer descriptions, which have limitations such as high subjectivity, low efficiency, and inconvenience of contact operation, making it difficult to meet the needs of large-scale screening or real-time monitoring. While subsequent methods using gaze heatmap analysis based on regions of interest have improved detection efficiency, this approach depends on manual annotation of stimuli, resulting in poor detection capabilities for dynamic test content and tasks. Furthermore, it only supports single task types and cannot cover multi-dimensional monitoring, leading to lower reliability of the assessment results. Summary of the Invention

[0004] To address the aforementioned technical problems, this specification provides one or more embodiments of a method, apparatus, and medium for detecting cognitive abnormalities based on pupil size.

[0005] One or more embodiments of this specification employ the following technical solutions: This specification provides one or more embodiments of a pupil-based cognitive abnormality detection method, applied to a cognitive detection system composed of an eye tracker, a display terminal, and a backend server. The method includes: The backend server determines the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; wherein, the detection task types include: attention-oriented detection type, attention-shifting detection type, and attention-maintaining detection type; The detection task content is transmitted to the display terminal for display, so as to receive eye movement flow data collected by the eye tracker during the display of the detection task content; The eye-movement flow data to be analyzed corresponding to each of the aforementioned detection task types is invoked, and the features to be detected are extracted from the corresponding eye-movement flow data based on the detection strategy corresponding to each of the aforementioned detection task types. The features to be detected are input into the corresponding type of pre-trained discriminant model to obtain the cognitive anomaly detection results for the current user.

[0006] Optionally, in one or more embodiments of this specification, the detection task content corresponding to different detection task types is determined based on the current user's identity information and current cognitive screening information, specifically including: The system receives the facial image features of the current user collected by the recognition module mounted on the display terminal, and compares the image feature vector corresponding to the facial image features with existing image feature vectors to achieve the identity verification of the current user. If the verification is successful, a user profile corresponding to the current user is constructed based on the current user's identity information and the historical cognitive anomaly detection records of the cognitive detection system. Based on the basic detection task content corresponding to the user profile, the personalized detection task content corresponding to the current cognitive screening information of the current user is supplemented to obtain the complete detection task content of the current user. Based on the monitoring tags corresponding to each part of the complete detection task content, the complete detection task content is broken down to determine the detection task content corresponding to different detection task types.

[0007] Optionally, in one or more embodiments of this specification, the detection task content is transmitted to the display terminal for display, so as to receive eye-tracking flow data collected by the eye tracker during the display of the detection task content, specifically including: Based on the hardware timestamps of the backend server and the display terminal, the latency data of the detection task content is determined; wherein, the latency data includes: transmission latency and rendering latency; Based on the task start time corresponding to the detection task content and the delay data, the effective acquisition time range of the eye tracker is determined; Acquire continuous eye movement flow data collected by the eye tracker within the effective acquisition time range; wherein, the continuous eye movement flow data includes: continuous eye movement points and continuous pupil diameter data; Based on the screen resolution of the display terminal and the distance between the current user's pupil and the display terminal, the viewing angle range of the current user is determined, and the corresponding viewing angle cone range is determined according to the viewing angle range, so as to merge the continuous eye movement points within the viewing angle cone range to obtain the sampled eye movement points; The sampled eye movement points and the related data corresponding to the sampled eye movement points are used as the eye movement flow data collected by the eye tracker.

[0008] Optionally, in one or more embodiments of this specification, the detection task content is transmitted to the display terminal for display. After receiving eye-tracking flow data collected by the eye tracker during the display of the detection task content, the method further includes: Obtain the identity-related field from the eye-tracking flow data, remove the identity-related field, and obtain the anonymized eye-tracking flow data; The desensitized eye movement flow data is normalized to obtain the eye movement flow data to be analyzed; Based on the acquisition time and detection task type corresponding to the eye movement flow data to be analyzed, the eye movement flow data to be analyzed is segmented. The eye movement flow data to be analyzed corresponding to odd-numbered acquisition time periods and the eye movement flow data to be analyzed corresponding to even-numbered acquisition time periods of the same detection task type are stored in different storage units of the distributed storage database.

[0009] Optionally, in one or more embodiments of this specification, based on the detection strategy corresponding to each of the detection task types, the extraction of features to be detected from the corresponding eye-tracking flow data to be analyzed specifically includes: If the detection task type is attention-oriented detection, then based on the fixation point of the eye movement flow data to be analyzed, the orientation transfer time used for the first visual cone range from the first target region to the second target region is obtained; wherein, the target objects corresponding to the first target region and the second target region are different; Based on the gaze point of the eye-tracking flow data to be analyzed, the time interval during which the current user's distance from the target point exceeds the range of the first viewpoint cone preset by the preset viewpoint is obtained during the process from the disappearance of the first target region to the appearance of the second target region, so as to use the appearance time interval as the orientation start time; A pupil coordinate system is established with time as the horizontal axis and the pupil diameter of the eye movement flow data to be analyzed as the vertical axis. The directional pupil integral from the appearance of the first target region to the disappearance of the second target region is calculated in the pupil coordinate system. The average values ​​of the directional transfer time, the directional start time, and the directional pupil integral are obtained respectively, and used as the features to be detected.

[0010] Optionally, in one or more embodiments of this specification, based on the detection strategy corresponding to each of the described detection task types, the extraction of features to be detected from the corresponding eye-tracking flow data to be analyzed specifically includes: If the detection task type is attention shift detection, based on the fixation point of the eye movement flow data to be analyzed, the transfer time used to transfer from the third target region to the first visual cone range of the second target region is obtained; wherein, the area of ​​the third target region is much larger than that of the second target region and the first target region; The time interval during which the distance between the current user and the target point exceeds the range of the first viewpoint cone preset by the preset viewpoint is obtained during the process from the disappearance of the third target area to the appearance of the second target area, so as to use the appearance time interval as the transfer start time; In the pupil coordinate system, calculate the transition pupil integral from the appearance of the third target region to the disappearance of the second target region; The average values ​​of the transfer time, the transfer initiation time, and the transfer pupil integral are obtained respectively, and used as the features to be detected.

[0011] Optionally, in one or more embodiments of this specification, based on the detection strategy corresponding to each of the described detection task types, the extraction of features to be detected from the corresponding eye-tracking flow data to be analyzed specifically includes: If the detection task type is attention maintenance detection type, then based on the fixation point of the eye movement flow data to be analyzed, the proportion of the fixation point in the target area and the proportion in the interference area are obtained respectively, and the fixation concentration ratio and interference offset ratio are determined. The gaze concentration ratio and the interference offset ratio are used as the features to be detected.

[0012] Optionally, in one or more embodiments of this specification, before inputting the feature to be detected into a pre-trained discriminant model of the corresponding type to obtain the cognitive anomaly detection result for the current user, the method further includes: Collect sample data from different test states to use as a training set; wherein, the different test states include: normal state, fatigue state, and cognitive abnormality state. Determine the autoencoder model corresponding to each detection task type, and train the autoencoder model corresponding to each detection task type based on the training set and the recognition result type corresponding to each detection task type to obtain the pre-trained discriminant model of the corresponding type.

[0013] This specification provides one or more embodiments of a pupil-based cognitive abnormality detection device, applied to a cognitive detection system based on an eye tracker, a display terminal, and a backend server. The device includes: The determining unit is used by the backend server to determine the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; wherein, the detection task types include: attention-oriented detection type, attention-shifting detection type, and attention-maintaining detection type; The acquisition unit is used to transmit the detection task content to the display terminal for display, and to receive eye movement flow data acquired by the eye tracker during the display of the detection task content; The extraction unit is used to call the eye movement flow data to be analyzed corresponding to each of the detection task types, so as to extract the features to be detected from the corresponding eye movement flow data to be analyzed based on the detection strategy corresponding to each of the detection task types. The discrimination unit is used to input the features to be detected into a pre-trained discrimination model of the corresponding type to obtain the cognitive anomaly detection result of the current user.

[0014] This specification provides one or more embodiments of a non-volatile computer storage medium storing computer-executable instructions, applied to a cognitive detection system based on an eye tracker, a display terminal, and a back-end server, wherein the computer-executable instructions are configured to execute any of the methods described above.

[0015] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: Based on user identity information and current cognitive screening information, the system determines the content of different detection tasks for each task type, fully considering individual differences and current cognitive status. This enables personalized detection plans and improves the targeting and effectiveness of the detection. Extracting corresponding features for different task types allows for a comprehensive assessment of the user's cognitive function from various perspectives, avoiding the bias that may result from a single task and making the results more comprehensive and accurate. Eye-tracking data collected by an eye tracker accurately reflects the user's cognitive behavior and physiological responses, providing a solid data foundation for cognitive abnormality detection. Pre-trained discriminant models based on corresponding types can quickly and accurately determine the user's cognitive state, improving the reliability and accuracy of the detection results. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A schematic flowchart of a pupil-based cognitive abnormality detection method provided in the embodiments of this specification; Figure 2 A schematic diagram of a pupil-based cognitive abnormality detection device provided in an embodiment of this specification; Figure 3 This is a schematic diagram of the structure of a non-volatile storage medium provided in the embodiments of this specification. Detailed Implementation

[0017] This specification provides a method, device, and medium for detecting cognitive abnormalities based on pupil size.

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] like Figure 1 As shown in the figure, this specification provides a schematic flowchart of a method for detecting cognitive abnormalities based on pupil size. Figure 1 As can be seen, in one or more embodiments of this specification, a pupil-based cognitive abnormality detection method is applied to a cognitive detection system composed of an eye tracker, a display terminal, and a backend server, and specifically includes the following steps: S101: The backend server determines the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; wherein, the detection task types include: attention-oriented detection type, attention-shifting detection type and attention-maintaining detection type.

[0020] In order to identify multiple types of cognitive abnormalities and thus determine attention-related cognitive abnormalities, in the embodiments of this specification, the backend server determines the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information. It should be noted that the detection task types include: attention orientation detection type, attention shift detection type, and attention maintenance detection type.

[0021] Specifically, in one or more embodiments of this specification, the detection task content corresponding to different detection task types is determined based on the current user's identity information and current cognitive screening information, specifically including the following process: To ensure the consistency of the detected objects and the traceability of the data, the system first receives the facial image features of the current user collected by the recognition module on the display terminal. The image feature vector corresponding to the facial image features is then compared with existing image feature vectors to verify the user's identity. Only after successful identity verification will subsequent cognitive screening operations for that user be initiated, preventing incorrect identity information from interfering with the normal progress of the detection task. If verification is successful, a user profile is constructed based on the current user's identity information and the historical cognitive anomaly detection records of the cognitive detection system. Based on the basic detection task content corresponding to the user profile, personalized detection task content corresponding to the current user's current cognitive screening information is supplemented to obtain the current user's complete detection task content. This achieves an organic combination of general detection foundations and individual differences, making the detection task more closely aligned with the current user's real-time cognitive state characteristics, improving detection accuracy and adaptability. Then, based on the monitoring tags corresponding to each part of the complete detection task content, the complete detection task content is broken down to determine the detection task content corresponding to different detection task types. The complete testing task content is broken down based on the monitoring tags, clarifying the corresponding content for different testing task types. This makes the entire testing task system well-organized, facilitating step-by-step operation during actual testing, improving the efficiency of testing work, and also helping to classify, statistically analyze, and perform different types of testing results later.

[0022] S102: The detection task content is transmitted to the display terminal for display, so as to receive eye movement flow data collected by the eye tracker during the display of the detection task content.

[0023] The detection task content obtained in step S101 is transmitted to the display terminal for display, so that the subject can view the visual stimulus items corresponding to the detection task content displayed on the display terminal. During the display of the detection task content, that is, during the subject's viewing, the eye tracker collects the subject's eye movement stream data and transmits it to the backend server for analysis.

[0024] Specifically, in one or more embodiments of this specification, the detection task content is transmitted to the display terminal for display, and during the display of the detection task content, eye movement flow data collected by the eye tracker is received, specifically including the following steps: First, to accurately grasp the time consumed by the detection task content from the backend server to the display terminal, and to provide a basis for subsequently determining the effective acquisition time range of the eye tracker, thereby reducing the analysis of easily manipulated eye-tracking data, this embodiment utilizes the hardware timestamps of the backend server and the display terminal to determine the latency data generated during the transmission and rendering of the detection task content, including transmission latency and rendering latency. Then, based on the task start time corresponding to the detection task content, combined with the previously determined latency data, the effective acquisition time range—the time interval in which the eye tracker can effectively acquire eye-tracking data—is clarified. By determining this effective acquisition time range, it is ensured that the acquired eye-tracking data is closely related to the actual display period of the detection task content, avoiding the acquisition and analysis of irrelevant or interfering data.

[0025] Then, continuous eye movement flow data collected by the eye tracker within the effective acquisition time range is acquired. It should be noted that the continuous eye movement flow data includes continuous eye movement points and continuous pupil diameter data. Next, to define the effective area of ​​the user's gaze and make subsequent processing of the eye movement point data more targeted, this embodiment determines the current user's visual field range based on the screen resolution of the display terminal and the distance between the user's pupil and the display terminal. Based on this visual field range, a corresponding visual field cone range is determined, and continuous eye movement points within the visual field cone range are merged to obtain sampled eye movement points. This merging process reduces the amount of data while retaining key gaze information, making the data more refined and easier for subsequent processing and analysis. Finally, the sampled eye movement points and their corresponding related data are used as the eye movement flow data collected by the eye tracker.

[0026] In one application scenario, the eye tracker samples at a frequency of 60Hz, providing continuous eye movement points and pupil diameter data. To compensate for jitter interference in the eye movement data caused by minor head movements or device errors, a viewing cone mechanism is introduced: continuous eye movement points within a certain range, such as a diameter of 300px, are grouped into a region of interest, and the geometric center of this cone represents the fixation point for that phase. Simultaneously, each cone is limited to a maximum of 10 eye movement points to balance accuracy and response sensitivity.

[0027] Furthermore, in one or more embodiments of this specification, after transmitting the detection task content to a display terminal for display, and after receiving eye-tracking flow data collected by the eye tracker during the display of the detection task content, the method further includes the following steps: First, to desensitize the eye-tracking stream data, the identity-related fields in the eye-tracking stream data are extracted and removed, resulting in desensitized eye-tracking stream data that does not contain personally identifiable information in subsequent processing. Then, to eliminate data differences caused by different acquisition devices and conditions, and to improve data consistency and comparability, the desensitized eye-tracking stream data in this embodiment is normalized to obtain the eye-tracking stream data to be analyzed. Based on the acquisition time and detection task type corresponding to the eye-tracking stream data to be analyzed, the data is fragmented. The eye-tracking stream data corresponding to odd-numbered acquisition time periods and even-numbered acquisition time periods of the same detection task type are stored in different storage units of a distributed storage database. Splitting eye-tracking data of the same task type into different storage units based on the parity of the acquisition time reduces the risk of single-point failure; for example, if one storage unit fails, the other part of the data can still completely reconstruct the entire task.

[0028] S103: Call the eye-movement flow data to be analyzed corresponding to each of the detection task types, and extract the features to be detected from the corresponding eye-movement flow data based on the detection strategy corresponding to each of the detection task types.

[0029] After obtaining the eye-tracking flow data collected by the eye tracker based on the above steps and storing it in a distributed storage database, it is also necessary to retrieve the eye-tracking flow data to be analyzed corresponding to each detection task type. This allows for the extraction of detectable features from the corresponding eye-tracking flow data based on the detection strategy corresponding to each detection task type. Specifically, in one or more embodiments of this specification, the extraction of detectable features from the corresponding eye-tracking flow data based on the detection strategy corresponding to each detection task type specifically includes: If the detection task type is attention-oriented detection, the orientation transfer time is obtained based on the fixation point of the eye movement flow data to be analyzed, taking into account the first visual cone range from the first target region to the second target region; where the target objects corresponding to the first and second target regions are different. Determining the orientation transfer time reflects the speed of attention orientation when the user moves from one target region to another. Also based on the fixation point of the eye movement flow data to be analyzed, the time interval between the disappearance of the first target region and the appearance of the second target region, during which the current user's distance from the target point exceeds the preset visual cone range, is obtained and used as the orientation initiation time. The orientation initiation time reflects the delay in initiating attention orientation when the target region changes. A pupil coordinate system is established with time as the horizontal axis and the pupil diameter of the eye movement flow data to be analyzed as the vertical axis. In this coordinate system, the orientation pupil integral is calculated during the time from the appearance of the first target region to the disappearance of the second target region. The average values ​​of the orientation transfer time, orientation initiation time, and orientation pupil integral are obtained respectively, and their average value is used as the feature to be detected.

[0030] By extracting three features—attention shift time, attention initiation time, and attention pupil integral—and calculating their average as the feature to be detected, a comprehensive characterization of a user's performance in attention orientation detection tasks can be achieved from multiple dimensions, including attention shift speed, initiation delay, and cognitive load. This more accurately reflects a user's attention orientation ability than a single feature, providing rich information for in-depth analysis of the user's cognitive behavior. Furthermore, attention shift time reflects the ability to quickly redirect attention, attention initiation time reflects the initiation characteristics of attention when the target changes, and attention pupil integral is related to mental effort and cognitive resource allocation during the attention process. Through comprehensive analysis of these features, changes in the user's cognitive behavior in attention orientation tasks can be captured more accurately, providing a strong basis for assessing the user's cognitive state and ability. Calculating the average of these three features as the feature to be detected reduces the random errors and fluctuations of single measurements to a certain extent, making the results more stable and reliable. At the same time, analyzing multiple features avoids bias and improves the accuracy and comprehensiveness of attention orientation detection.

[0031] In one application scenario, if the detection task is an attention-oriented detection type, a standard orientation attention experiment paradigm can be used. Each stimulus consists of a central fixation point ("+") followed by peripheral cues ("."). The three sets of images are presented sequentially, each lasting 3 seconds. The orientation transfer time is determined by the time taken to move from the last fixation point in the "+" region to the first visual cone in the "." region. The orientation initiation time is determined by the time interval between the disappearance of the "+" image and the appearance of the "." image, and the appearance of the first visual cone at a fixation point more than 75px from the target point. The integral value within a 6-second period from the appearance of the "+" to the disappearance of the "." is calculated, measuring the subject's cognitive engagement and determining the orientation pupil integral. The average values ​​of the three sets are calculated for each indicator, yielding the average orientation transfer time, average orientation initiation time, and average orientation pupil integral as the features to be detected.

[0032] Specifically, in one or more embodiments of this specification, based on the detection strategy corresponding to each detection task type, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, which specifically includes the following process: When the detection task is attention shift detection, the transfer time from the fixation point in the eye movement flow data to the first visual cone range of the second target region is obtained. It should be noted that, to better study the attention shift characteristics between target regions of different sizes, the area of ​​the third target region is set to be much larger than that of the second and first target regions. Also based on the fixation point in the eye movement flow data, the time interval between the disappearance of the third target region and the appearance of the second target region, during which the current user's distance from the target point exceeds the preset visual cone range, is obtained and used as the transfer initiation time. This reflects the delay in initiating attention shift when the user switches target regions. In the previously established pupil coordinate system, the transfer pupil integral is calculated during the time from the appearance of the third target region to the disappearance of the second target region. By analyzing the change in pupil diameter over time, the cognitive load or mental effort of the user during attention shift can be understood. Then, the average values ​​of the transfer time, transfer initiation time, and transfer pupil integral are obtained, and these average values ​​are used as the features to be detected.

[0033] In one application scenario, when the detection task is an attention shift detection type, this test assesses the ability to switch attention from a salient target to a cueing stimulus against a background of visual interference. Each stimulus set contains a face image and a "." cue point. The face image does not disappear when the "." image appears, constituting a distraction and testing the subject's attention disengagement ability. The time from the last fixation point in the face region to the first visual cone in the "." region is determined as the shift time. The delay from the disappearance of the face image to the appearance of the "." point and its fixation within a range greater than 75 pixels is determined as the shift initiation time. The pupillary integral of the shift is determined by the pupillary change over 6 seconds from the appearance of the face image to the end of the ".". The average of the three sets is used as the feature to be detected.

[0034] Specifically, in one or more embodiments of this specification, based on the detection strategy corresponding to each detection task type, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: When the detection task is attention maintenance detection, based on the fixation points in the eye movement flow data to be analyzed, the proportion of fixation points located in the target region and the proportion located in the distracting region are calculated. These two proportions are used to determine the fixation concentration ratio and the distraction shift ratio. This reflects the user's level of focus on the target region and the degree to which they are attracted by distracting regions during the attention maintenance task.

[0035] In one application scenario, when the detection task is an attention maintenance test, the ability of a subject to maintain sustained attention to a specific target is assessed. Two types of regions appear during the test: a white, stationary dot region (the target area) and surrounding flashing dot regions (interference areas). Ideally, the subject should maintain continuous attention to the white dot region.

[0036] S104: Input the feature to be detected into the corresponding type of pre-trained discriminant model to obtain the cognitive anomaly detection result of the current user.

[0037] After obtaining the features to be detected based on the above step S103, the obtained features to be detected are input into the corresponding type of pre-trained discrimination model to obtain the cognitive anomaly detection result of the current user.

[0038] Furthermore, in one or more embodiments of this specification, before inputting the feature to be detected into a pre-trained discriminant model of the corresponding type to obtain the cognitive anomaly detection result for the current user, the method further includes: Sample data from different test states, including normal, fatigued, and cognitively abnormal states, are collected to form a training set. Integrating this data ensures the training set covers a wide range of possible user states, providing a rich data foundation for subsequent model training. Then, the corresponding autoencoder model for each detection task type is determined. Based on the training set and the corresponding recognition result types, the autoencoder model for each detection task type is trained to obtain a pre-trained discriminant model for that type. The autoencoder model for each detection task type is determined and trained specifically based on the training set and the corresponding recognition result types. Since different detection task types may have their own unique characteristics and focuses, the pre-trained discriminant model trained in this way can better adapt to the needs of specific detection task types, thereby improving the accuracy of cognitive abnormality detection when analyzing the features to be detected for that task type, and providing users with specific and accurate detection results.

[0039] In one application scenario, during sample data collection, the system not only records the eye movement and pupillary characteristics of subjects under three tasks, but also labels the status of some volunteers based on the actual situation. These include: volunteers in a normal state (no fatigue, no cognitive impairment); volunteers in a fatigued state (whose status was artificially adjusted before the test, such as sleep deprivation or cognitive load tasks); and volunteers with cognitive abnormalities (those with diagnosed cognitive impairments such as mild cognitive impairment or attention deficit). These three types of data together constitute the model training set. To adapt to scenarios lacking anomaly sample labeling in actual deployments, the AutoEncoder model can be used as the core method for constructing the anomaly detection model. The specific method is as follows: During the training phase, the autoencoder is trained using data from volunteers already identified as "normal," enabling it to learn low-dimensional representations and reconstruction mechanisms of normal feature patterns. In the detection phase, the system reconstructs data from new subjects and calculates the reconstruction error. The judgment criterion is: if the reconstruction error exceeds a preset threshold, which is determined with the assistance of samples from fatigued and cognitively abnormal volunteers, then cognitive abnormalities are considered to exist in that category of attention task. Then, separate Autoencoder models are built for each of the three task categories to improve the specificity and accuracy of detection. Finally, the system will output the following anomaly identification results based on the three tasks: whether attention orientation ability is abnormal and its severity; whether attention shifting ability is abnormal and its resistance to interference; and whether attention maintenance ability is abnormal and its concentration stability assessment.

[0040] like Figure 2 As shown in the diagram, this specification provides a schematic diagram of a pupil-based cognitive abnormality detection device. Figure 2As can be seen, in one or more embodiments of this specification, a pupil-based cognitive abnormality detection device is applied to a cognitive detection system composed of an eye tracker, a display terminal, and a backend server. The device includes: The determining unit is used by the backend server to determine the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; wherein, the detection task types include: attention-oriented detection type, attention-shifting detection type, and attention-maintaining detection type; The acquisition unit is used to transmit the detection task content to the display terminal for display, and to receive eye movement flow data acquired by the eye tracker during the display of the detection task content; The extraction unit is used to call the eye movement flow data to be analyzed corresponding to each of the detection task types, so as to extract the features to be detected from the corresponding eye movement flow data to be analyzed based on the detection strategy corresponding to each of the detection task types. The discrimination unit is used to input the features to be detected into a pre-trained discrimination model of the corresponding type to obtain the cognitive anomaly detection result of the current user.

[0041] like Figure 3 As shown in the diagram, this specification provides a schematic diagram of the structure of a non-volatile storage medium. Figure 3 As can be seen, in one or more embodiments of this specification, a non-volatile storage medium stores computer-executable instructions 301, which are applied to a cognitive detection system based on an eye tracker, a display terminal, and a back-end server. The computer-executable instructions 301 are capable of executing any of the methods described above.

[0042] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0043] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0044] The above description is merely one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to the one or more embodiments of this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims

1. A pupil-based method for detecting cognitive abnormalities, applied to a cognitive detection system consisting of an eye tracker, a display terminal, and a backend server, characterized in that, The method includes: The backend server determines the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; wherein, the detection task types include: attention-oriented detection type, attention-shifting detection type, and attention-maintaining detection type; The detection task content is transmitted to the display terminal for display, so as to receive eye movement flow data collected by the eye tracker during the display of the detection task content; The eye-movement flow data to be analyzed corresponding to each of the aforementioned detection task types is invoked, and the features to be detected are extracted from the corresponding eye-movement flow data based on the detection strategy corresponding to each of the aforementioned detection task types. The features to be detected are input into a pre-trained discriminant model of the corresponding type to obtain the cognitive anomaly detection results of the current user. Based on the detection strategy corresponding to each of the aforementioned detection task types, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: If the detection task type is attention-oriented detection, then based on the fixation point of the eye movement flow data to be analyzed, the orientation transfer time used for the first visual cone range from the first target region to the second target region is obtained; wherein, the target objects corresponding to the first target region and the second target region are different; Based on the gaze point of the eye-tracking flow data to be analyzed, the time interval during which the current user's distance from the target point exceeds the range of the first viewpoint cone of the preset viewpoint during the process from the disappearance of the first target region to the appearance of the second target region is obtained, and the appearance time interval is used as the orientation start time. A pupil coordinate system is established with time as the horizontal axis and the pupil diameter of the eye movement flow data to be analyzed as the vertical axis. The directional pupil integral from the appearance of the first target region to the disappearance of the second target region is calculated in the pupil coordinate system. The average values ​​of the directional transfer time, the directional start time, and the directional pupil integral are obtained respectively, and used as the features to be detected; Based on the detection strategy corresponding to each of the aforementioned detection task types, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: If the detection task type is attention shift detection, based on the fixation point of the eye movement flow data to be analyzed, the transfer time used to transfer from the third target region to the first visual cone range of the second target region is obtained; wherein, the area of ​​the third target region is much larger than that of the second target region and the first target region; The time interval during which the distance between the current user and the target point exceeds the range of the first viewpoint cone of the preset viewpoint during the process from the disappearance of the third target area to the appearance of the second target area is obtained, and the appearance time interval is used as the transfer start time. In the pupil coordinate system, calculate the transition pupil integral from the appearance of the third target region to the disappearance of the second target region; The average values ​​of the transfer time, the transfer initiation time, and the transfer pupil integral are obtained respectively, and used as the features to be detected; Based on the detection strategy corresponding to each of the aforementioned detection task types, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: If the detection task type is attention maintenance detection type, then based on the fixation point of the eye movement flow data to be analyzed, the proportion of the fixation point in the target area and the proportion in the interference area are obtained respectively, and the fixation concentration ratio and interference offset ratio are determined. The gaze concentration ratio and the interference offset ratio are used as the features to be detected.

2. The method for detecting cognitive abnormalities based on pupil size according to claim 1, characterized in that, Based on the current user's identity information and current cognitive screening information, the detection task content corresponding to different detection task types is determined, specifically including: The system receives the facial image features of the current user collected by the recognition module mounted on the display terminal, and compares the image feature vector corresponding to the facial image features with existing image feature vectors to achieve the identity verification of the current user. If the verification is successful, a user profile corresponding to the current user is constructed based on the current user's identity information and the historical cognitive anomaly detection records of the cognitive detection system. Based on the basic detection task content corresponding to the user profile, the personalized detection task content corresponding to the current cognitive screening information of the current user is supplemented to obtain the complete detection task content of the current user. Based on the monitoring tags corresponding to each part of the complete detection task content, the complete detection task content is broken down to determine the detection task content corresponding to different detection task types.

3. The method for detecting cognitive abnormalities based on pupil size according to claim 1, characterized in that, The detection task content is transmitted to the display terminal for display, and during the display of the detection task content, eye movement flow data collected by the eye tracker is received, specifically including: Based on the hardware timestamps of the backend server and the display terminal, the latency data of the detection task content is determined; wherein, the latency data includes: transmission latency and rendering latency; Based on the task start time corresponding to the detection task content and the time delay data, the effective acquisition time range of the eye tracker is determined; Acquire continuous eye movement flow data collected by the eye tracker within the effective acquisition time range; wherein, the continuous eye movement flow data includes: continuous eye movement points and continuous pupil diameter data; Based on the screen resolution of the display terminal and the distance between the current user's pupil and the display terminal, the viewing angle range of the current user is determined, and the corresponding viewing angle cone range is determined according to the viewing angle range, so as to merge the continuous eye movement points within the viewing angle cone range to obtain the sampled eye movement points; The sampled eye movement points and the corresponding related data are used as the eye movement flow data collected by the eye tracker.

4. The method for detecting cognitive abnormalities based on pupil size according to claim 1, characterized in that, The method further includes transmitting the detection task content to the display terminal for display, and receiving eye-tracking flow data collected by the eye tracker during the display of the detection task content. Obtain the identity-related field from the eye-tracking flow data, remove the identity-related field, and obtain the anonymized eye-tracking flow data; The desensitized ocular flow data is normalized to obtain the ocular flow data to be analyzed; Based on the acquisition time and detection task type corresponding to the eye movement flow data to be analyzed, the eye movement flow data to be analyzed is segmented. The eye movement flow data to be analyzed corresponding to odd-numbered acquisition time periods and the eye movement flow data to be analyzed corresponding to even-numbered acquisition time periods of the same detection task type are stored in different storage units of the distributed storage database.

5. The method for detecting cognitive abnormalities based on pupil size according to claim 1, characterized in that, Before inputting the features to be detected into a pre-trained discriminant model of the corresponding type to obtain the cognitive anomaly detection result for the current user, the method further includes: Collect sample data from different test states to use as a training set; wherein, the different test states include: normal state, fatigue state, and cognitive abnormality state. Determine the autoencoder model corresponding to each detection task type, and train the autoencoder model corresponding to each detection task type based on the training set and the recognition result type corresponding to each detection task type to obtain the pre-trained discriminant model of the corresponding type.

6. A pupil-based cognitive abnormality detection device, applied to a cognitive detection system consisting of an eye tracker, a display terminal, and a backend server, characterized in that, The device includes: The determining unit is used by the backend server to determine the detection task content corresponding to different detection task types based on the current user's identity information and current cognitive screening information; wherein, the detection task types include: attention-oriented detection type, attention-shifting detection type, and attention-maintaining detection type; The acquisition unit is used to transmit the detection task content to the display terminal for display, and to receive eye movement flow data acquired by the eye tracker during the display of the detection task content; The extraction unit is used to call the eye movement flow data to be analyzed corresponding to each of the detection task types, so as to extract the features to be detected from the corresponding eye movement flow data to be analyzed based on the detection strategy corresponding to each of the detection task types. The discrimination unit is used to input the features to be detected into a pre-trained discrimination model of the corresponding type to obtain the cognitive anomaly detection result of the current user; Based on the detection strategy corresponding to each of the aforementioned detection task types, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: If the detection task type is attention-oriented detection, then based on the fixation point of the eye movement flow data to be analyzed, the orientation transfer time used for the first visual cone range from the first target region to the second target region is obtained; wherein, the target objects corresponding to the first target region and the second target region are different; Based on the gaze point of the eye-tracking flow data to be analyzed, the time interval during which the current user's distance from the target point exceeds the range of the first viewpoint cone of the preset viewpoint during the process from the disappearance of the first target region to the appearance of the second target region is obtained, and the appearance time interval is used as the orientation start time. A pupil coordinate system is established with time as the horizontal axis and the pupil diameter of the eye movement flow data to be analyzed as the vertical axis. The directional pupil integral from the appearance of the first target region to the disappearance of the second target region is calculated in the pupil coordinate system. The average values ​​of the directional transfer time, the directional start time, and the directional pupil integral are obtained respectively, and used as the features to be detected; Based on the detection strategy corresponding to each of the aforementioned detection task types, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: If the detection task type is attention shift detection, based on the fixation point of the eye movement flow data to be analyzed, the transfer time used to transfer from the third target region to the first visual cone range of the second target region is obtained; wherein, the area of ​​the third target region is much larger than that of the second target region and the first target region; The time interval during which the distance between the current user and the target point exceeds the range of the first viewpoint cone of the preset viewpoint during the process from the disappearance of the third target area to the appearance of the second target area is obtained, and the appearance time interval is used as the transfer start time. In the pupil coordinate system, calculate the transition pupil integral from the appearance of the third target region to the disappearance of the second target region; The average values ​​of the transfer time, the transfer initiation time, and the transfer pupil integral are obtained respectively, and used as the features to be detected; Based on the detection strategy corresponding to each of the aforementioned detection task types, the corresponding eye movement flow data to be analyzed is used to extract the features to be detected, specifically including: If the detection task type is attention maintenance detection type, then based on the fixation point of the eye movement flow data to be analyzed, the proportion of the fixation point in the target area and the proportion in the interference area are obtained respectively, and the fixation concentration ratio and interference offset ratio are determined. The gaze concentration ratio and the interference offset ratio are used as the features to be detected.

7. A non-volatile storage medium storing computer-executable instructions, applied to a cognitive detection system based on an eye tracker, a display terminal, and a backend server, characterized in that... The computer-executable instructions are capable of performing the method described in any one of claims 1-5.

Images