Collaborative interaction method, terminal and system for collecting eye movement information, brain fnirs detection data, and electroencephalogram data

Abstract

The application provides a collaborative interaction method, terminal and system for collecting eye movement information, brain fnirs detection data and electroencephalogram data. The method comprises sequentially performing a preset number of task blocks at a preset first time interval for a subject by using a display, and pseudo-randomly alternately performing a positive saccade trial and a negative saccade trial in each task block. The positive saccade trial comprises sequentially presenting a first visual element, a first mark indicating a positive saccade and a second visual element. The negative saccade trial comprises sequentially presenting the first visual element, a second mark indicating a negative saccade and the second visual element. The eye movement information, the brain fnirs detection data and the electroencephalogram data of the subject in each trial are collected. Thus, the obtained eye movement information, brain fnirs detection data and electroencephalogram data can be used for auxiliary evaluation of brain function related to cognitive dysfunction, and a more accurate classification model can be established for subsequent diagnosis of cognitive dysfunction.

Description

Technical Field

[0001] The present application relates to the technical field of assisted brain function assessment, and specifically to a collaborative interaction method, terminal, and system for collecting eye movement information, brain fNIRS detection data, and electroencephalogram data. Background Art

[0002] Cognitive dysfunction, including schizophrenia (SZ), bipolar disorder (BD), major depressive disorder (MDD), and attention deficit hyperactivity disorder (ADHD), presents with diverse and complex clinical symptoms. Physicians often rely on their personal clinical experience to distinguish these symptoms. As the number of patients continues to increase, the pressure on physicians to diagnose them also increases. To alleviate this problem, researchers have developed a variety of brain function assessment methods to help physicians speed up diagnosis.

[0003] Among the related technologies, functional near-infrared spectroscopy (fNIRS) has good spatial resolution, electroencephalogram (EEG) technology has excellent temporal resolution, and eye tracking technology can provide important information about visual attention and cognitive processing. Although using a single technology to assist in the assessment of brain function is helpful in distinguishing the types of cognitive dysfunction, the clinical symptoms of different cognitive dysfunctions may be similar when evaluated using a single technology, making it impossible to accurately distinguish different types of cognitive dysfunction.

[0004] The blood oxygen signal collected by fNIRS is a slow signal, the EEG signal collected by EEG is a fast signal, and the eye movement information collected by eye tracking technology is a rapidly changing behavioral signal. Therefore, the method suitable for collecting fNIRS signals is usually not suitable for collecting EEG signals and eye movement information. To obtain accurate evaluation indicators in different dimensions, multiple brain function auxiliary assessments are required, which brings inconvenience to the subjects. Therefore, there is still much room for improvement in the technology that combines data from multiple modalities to perform auxiliary brain function assessments. Summary of the Invention

[0005] In view of the above technical problems in the prior art, the present application is proposed. The present application aims to provide a collaborative interaction method for collecting eye movement information, brain fnirs detection data and electroencephalogram data, which can provide a three-modal task paradigm suitable for simultaneously collecting effective eye movement information, brain fnirs detection data and electroencephalogram data, and can realize intuitive, convenient and efficient collaborative interaction with the subject using a single display. In the process of collaborative interaction, eye movement information, brain fnirs detection data and electroencephalogram data of different time periods are obtained. The obtained eye movement information, brain fnirs detection data and electroencephalogram data can objectively and comprehensively reflect the influence of different brain function conditions of the subject on each modal data, and facilitate the establishment of a more accurate classification model for subsequent diagnosis of cognitive dysfunction.

[0006] According to the first aspect of the present application, a collaborative interaction method for collecting eye movement information, brain fnirs detection data and electroencephalogram data is provided. The collaborative interaction method includes using a display to sequentially perform a preset number of task blocks on a subject at a preset first time interval, and pseudo-randomly alternatingly performing a positive saccade trial and a negative saccade trial in each task block. Performing the positive saccade trial includes sequentially presenting a first visual element for a preset first time period, presenting a first identifier indicating a positive saccade for a preset second time period, and presenting a second visual element for a preset third time period. Performing the negative saccade trial includes sequentially presenting a first visual element for a preset first time period, presenting a second identifier indicating a negative saccade for a preset second time period, and presenting a second visual element for a preset third time period. The method also includes collecting eye movement information of the subject in the preset third time period of each trial, collecting brain fnirs detection data during the first time interval and in each task block, and collecting electroencephalogram data in the preset third time period before and after a time period less than the first time period in each trial.

[0007] According to a second embodiment of the present application, a terminal for providing a trimodal paradigm of eye movement information, fNIRS brain detection data, and electroencephalogram (EEG) data is provided. The terminal is configured to use a processor to: sequentially execute a preset number of task blocks for a subject at a preset first time interval, and pseudo-randomly alternately execute positive saccade trials and antisaccade trials in each task block, wherein executing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first identifier indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing the antisaccade trial includes sequentially presenting the first visual element within a preset first time period, presenting a second identifier indicating an antisaccade within a preset second time period, and presenting the second visual element within a preset third time period; wherein the fNIRS brain detection data during the first time interval and in each task block, and the EEG data within a preset third time period from a duration less than the first time period before to after the second time period in each trial, are used to collaboratively analyze with the subject's eye movement information within the preset third time period in each trial to obtain an auxiliary brain function assessment result of the subject.

[0008] According to the third embodiment of the present application, a data acquisition system is provided. The data acquisition system includes an eye movement information acquisition device, an fNIRS device, and an EEG device. Each device is used to execute the collaborative interaction method for collecting eye movement information, brain fNIRS detection data and EEG data as described in each embodiment of the present application.

[0009] According to a fourth embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data as described in the various embodiments of the present application is implemented.

[0010] According to a fifth embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data is provided via a processor. Specifically, the three-modal paradigm includes sequentially executing a preset number of task blocks for the subject at a preset first time interval, and pseudo-randomly alternatingly executing positive saccade trials and antisaccade trials in each task block, wherein executing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first identifier indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing the antisaccade trial includes sequentially presenting the first visual element within the preset first time period, presenting a second identifier indicating an antisaccade within the preset second time period, and presenting the second visual element within the preset third time period.

[0011] Compared with the prior art, the embodiments of the present application have the following advantages:

[0012] The collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data provided in the embodiment of the present application uses a display to sequentially execute a preset number of task blocks for the subject at a preset first time interval, and pseudo-randomly alternately executes positive saccade trials and anti-saccade trials in each task block, presents a first visual element in a first time period of each trial, presents an indicator mark in a second time period, and presents a second visual element in a third time period; in this way, for brain fNIRS detection data, since the task block contains positive saccade trials and anti-saccade trials, it is sufficient to collect effective and differentiated brain fNIRS detection data, and each task block The design of the time intervals between the two trials meets the time requirements for fNIRS signal acquisition and analysis, thereby obtaining high-quality fNIRS brain data. Regarding EEG data, the pseudo-random alternation of positive and negative saccade trials effectively reduces the subject's prediction and practice effects on the stimulus, ensuring that each stimulus triggers a maximal EEG response. Regarding eye movement information, different types of trials can meet the needs of assessing brain function in different dimensions while also meeting the signal acquisition requirements of EEG equipment and the eye movement information collection requirements. The eye movement information obtained from positive and negative saccade trials can effectively assess and verify brain functions related to cognitive dysfunction. Therefore, while performing the aforementioned task blocks, subjects can simultaneously acquire high-quality trimodal data: eye movement, fNIRS, and EEG. The data collected from each modality can be used to assist in the assessment of brain function related to cognitive dysfunction.

[0013] In this way, based on the collaborative interaction method provided by the present application, a trimodal task paradigm can be provided for the subject, so that the subject can simultaneously obtain effective and accurate eye movement information, brain fNIRS detection data and EEG data while executing the trimodal task paradigm provided by the embodiment of the present application.

[0014] This allows for accurate assessment of brain function in subjects, especially those with mental illness or a tendency toward mental illness. It also makes it easier for doctors to differentiate between patients with mental disorders based on the objective biological indicators obtained from the assessment, and facilitates the establishment of a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

[0015] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar components in different views. Similar reference numerals with letter suffixes or different letter suffixes may represent different examples of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not by way of limitation, and together with the description and claims, serve to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary and are not intended to be exhaustive or exclusive embodiments of the present method, apparatus, system, or non-transitory computer-readable medium having instructions for implementing the method.

[0017] Figure 1 A flowchart of a collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data provided according to an embodiment of the present application is shown.

[0018] Figure 2 A schematic diagram of displaying various task blocks based on a display according to an embodiment of the present application is shown.

[0019] Figure 3 A schematic diagram of a terminal for providing a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data according to an embodiment of the present application is shown.

[0020] Figure 4 A schematic diagram of a data acquisition system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific embodiments, but are not intended to limit the present application.

[0022] The words "first", "second" and similar terms used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar terms used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The steps in the execution order can be combined, decomposed, or swapped, as long as the logical relationship of the execution content is not affected.

[0023] All terms used in this application (including technical or scientific terms) have the same meaning as understood by a person of ordinary skill in the art to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in common dictionaries, for example, should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein. Techniques and devices known to a person of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such techniques and devices should be considered part of the specification.

[0024] Figure 1 A flowchart of a collaborative interaction method for collecting eye movement information, fNIRS brain detection data, and EEG data provided in accordance with an embodiment of the present application is shown. Specifically, the method shown in steps S101 to S102 is executed. In this embodiment, the arrows shown in the figure for each step are merely examples of the execution order, not limitations. The technical solution of the present application is not limited to the execution order described in the embodiment. The steps in the execution order can be combined, decomposed, or swapped, as long as the logical relationship of the execution content is not affected.

[0025] For example, before the subject performs the trimodal task paradigm, the subject can be equipped with an eye tracker suitable for collecting eye movement information and a head cap for collecting fNIRS brain data and EEG data. The installation position of the fNIRS signal acquisition probe and the installation position of the EEG electrodes configured in the head cap can be set by the user based on prior medical knowledge and are not specifically limited. The light source in the eye tracker will not interfere with the collection of fNIRS brain data and EEG data.

[0026] After the subject wears the eye tracker suitable for collecting eye movement information and the head cap for collecting brain fNIRS detection data and EEG data, the trimodal task can be started to simultaneously collect the subject's eye movement information, brain fNIRS detection data and EEG data while performing the trimodal task.

[0027] Specifically, in step S101, a display is used to sequentially execute a preset number of task blocks for the subject at a preset first time interval, and positive saccade trials and antisaccade trials are pseudo-randomly alternatingly executed in each task block. Executing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first identifier indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing the antisaccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a second identifier indicating an antisaccade within a preset second time period, and presenting a second visual element within a preset third time period.

[0028] Specifically, a task prompt requiring the subject to perform a trimodal task can be displayed on the screen of the display, so that the subject performs the corresponding trimodal task according to the task prompt displayed on the display, thereby simultaneously collecting the subject's eye movement information, brain fNIRS detection data, and EEG data when performing the trimodal task. Based on the collaborative interaction method provided in the embodiment of the present application, the acquired eye movement information, brain fNIRS detection data, and EEG data are more accurate and reliable, and can objectively and comprehensively reflect the impact of the subject's different brain function conditions on each modal data, and can be used as effective data for auxiliary analysis of the subject's brain function condition.

[0029] like Figure 2 As shown, before the first task block 201 is presented, a cross symbol is displayed on the screen to prompt the subject to take a break before executing the first task block 201. This allows the subject to be aware of the upcoming task and to be mentally prepared to execute the task. In other words, in some other embodiments, before executing the first task block, a second time interval is executed, and the duration of the second time interval is 10s-60s. This helps the subject to rest during the second time interval, adjust their state, and concentrate on executing subsequent tasks.

[0030] After the subject completes the task of task block 201, he / she rests for a first time interval, then continues to perform the task of task block 201, rests for a first time interval again, and then continues to perform the task of task block 201, and so on, until the subject completes the preset number of task blocks 201. For example, the first time interval may be 20 seconds, and the second time interval may also be 20 seconds, that is, Figure 2 The first time interval shown may be the same as the second time interval. In addition, after the subject completes each task block 201, the subject will take a rest according to the first time interval, thereby reducing the task load of the subject.

[0031] Figure 2 Only the first task block 201 and the first time interval adjacent to the first task block 201 are shown, and the remaining task blocks 201 and the first time intervals adjacent to the first task block 201 are all represented by ellipsis, and, Figure 2 The content shown is only for facilitating the understanding of the technical solution and does not constitute a limitation on the specific solution.

[0032] The first time interval may also be other time periods such as 30s, 40s, etc. The preset number of task blocks 201 is at least 2, which can be specifically set by doctors or relevant researchers based on medical experience.

[0033] In a preferred embodiment of the present application, for a complete task, 2-8 task blocks are sequentially executed with a first time interval of 10s-60s, and the duration of each task block is 10s-60s. Moreover, the duration of each task block is consistent. Setting the first time interval to 10s-60s is conducive to blood oxygen level recovery, while alleviating the task load of the subject, and not causing the overall task duration to be too long due to the long rest time interval. Moreover, 2-8 task blocks are sequentially executed with a duration of 10s-60s for each task block, which can cause a good fNIRS signal response, can meet the extraction and analysis requirements of collecting brain fNIRS detection data, and obtain accurate and effective brain fNIRS detection data.

[0034] like Figure 2 , pseudo-randomly perform positive saccade trials 202 and antisaccade trials 203 alternately in each task block 201, assuming Figure 2 A task block 201 shown includes two positive saccade trials 202 and three antisaccade trials 203. The subject can first perform a positive saccade trial 202, then a antisaccade trial 203, then a antisaccade trial 203, then a positive saccade trial 202, and then a antisaccade trial 203. Alternatively, the subject can first perform a antisaccade trial 203, then a positive saccade trial 202, then a antisaccade trial 203, then a antisaccade trial 203, and then a positive saccade trial 202.

[0035] That is to say, the order of alternating presentation of positive saccade trials and anti-saccade trials in each task block can be random, which helps to avoid the subject's anticipation of the subsequent saccade movements.

[0036] In this embodiment, by pseudo-randomly alternating presentation of positive saccade trials and antisaccade trials and repeatedly executing each task block multiple times, the fNIRS signal response in the brain can be better induced, and the EEG response that meets the usage requirements can be collected.

[0037] This is only an illustrative example and does not constitute a limitation on the specific solution. Figure 2 As shown, the background color of each screen displayed is black, that is, Figure 2 A black screen is used as an example for explanation.

[0038] like Figure 2In a positive saccade trial 202, a white dot in the center of a black screen, a five-pointed star pattern in the center of a black screen, and a gray dot at the edge of the black screen can be displayed in sequence. Displaying these three screens in sequence completes a positive saccade trial 202. The task block 201 only shows one positive saccade trial 202; the remaining positive saccade trials 202 are indicated by ellipsis.

[0039] In a positive saccade trial 202, the first visual element 204 is a white dot in the center of a black screen, the first logo 205 is a five-pointed star pattern in the center of the black screen, and the second visual element 206 is a gray dot at the edge of the black screen. Furthermore, in a positive saccade trial 202, the first visual element 204 is presented in the first time period, the first logo 205 is presented in the second time period, and the second visual element 206 is presented in the third time period.

[0040] In an antisaccade trial 203, a white dot in the center of a black screen, a heart-shaped pattern in the center of a black screen, and a gray dot at the edge of the black screen can be displayed in sequence. Displaying these three screens in sequence completes an antisaccade trial 203. The task block 201 only shows one antisaccade trial 203; the remaining antisaccade trials 203 are indicated by ellipsis.

[0041] In antisaccade trial 203, first visual element 204 is a white dot in the center of a black screen, second identifier 207 is a heart-shaped pattern in the center of the black screen, and second visual element 206 is a gray dot at the edge of the black screen. Furthermore, in antisaccade trial 203, first visual element 204 is presented in the first time period, second identifier 207 is presented in the second time period, and second visual element 206 is presented in the third time period.

[0042] Then, in some other embodiments, in each trial, the first time period is 200ms-2000ms, the second time period is 500ms-2000ms, and the third time period is 1000ms-2500ms. For example, Figure 2 The first time period shown in FIG can be 0.2s, 0.5s, 0.8s, 1s, 1.3s, 1.5s, or 1.7s, the second time period can be 0.5s, 0.9s, 1.3s, 1.5s, 1.7s, or 1.9s, and the third time period can be 1s, 1.3s, 1.7s, 2s, 2.2s, or 2.4s. That is, the time it takes for the subject to perform a positive saccade trial 202 or an antisaccade trial 203 is no less than 1.7s, which is moderately difficult for the subject. In this way, the difficulty of the task performed by the subject can be well controlled, so that the subject can well complete the entire trimodal task.

[0043] In this embodiment, the first visual element, the indicator mark and the second visual element are presented in different time periods and on different screens. By reasonably setting different time periods for executing tasks, especially the third time period being longer, free eye movements can be effectively controlled and the overlap of brain reactions caused by stimulation tasks can be reduced, and interference such as artifacts related to eye movements can be reduced, which is conducive to collecting accurate and effective eye movement information and EEG data.

[0044] In some other embodiments of the present application, the first visual element and the second visual element have different appearance features, and the first logo and the second logo have different appearance features. The appearance features may be one or more of color features, shape features, and size features, and are not limited thereto, as long as the first visual element and the second visual element can be distinguished by the appearance features, and the first logo and the second logo can be distinguished by the appearance features. Figure 2 The first mark 205 and the second mark 207 have different shape features, and the first visual element 204 and the second visual element 206 have different color features.

[0045] In some other embodiments of the present application, in the positive saccade trial, the first identifier is presented to instruct the subject to perform an action of gazing at the location of the second visual element within a preset third time period. Figure 2 In the positive saccade trial 202, when the five-pointed star pattern is presented on the black screen in the second time period, it means that the subject has to look at the position of the gray dot on the next black screen in the third time period after the five-pointed star pattern disappears, that is, the five-pointed star pattern is used to instruct the subject to perform the action of looking at the position of the gray dot in the next time period.

[0046] In antisaccade trial 203, when the heart-shaped pattern is presented on the black screen during the second time period, it means that after the heart-shaped pattern disappears, the subject should fixate on a position equidistant from the gray dot on the black screen during a third time period. That is, the heart-shaped pattern instructs the subject to fixate on a position equidistant from the gray dot during the next time period. In other words, in the antisaccade trial, the second indicator is presented to instruct the subject to fixate on a position equidistant from the second visual element during the preset third time period.

[0047] In addition, the first identifier and the second identifier can be set according to relevant information such as the age or mental condition of the subject. For example, if the subject is a child, the first identifier and the second identifier can be set as cartoon symbols to make the child subject interested in the task to be performed.

[0048] The subject can be allowed to perform an exercise task before formally starting to perform the three-modal task paradigm, so that the subject knows to perform the action of gazing at the position of the second visual element when seeing the first mark, and to perform the action of gazing at the position opposite to the position of the second visual element at an equal distance when seeing the second mark, so that the subject can perform the task of each task block based on the task prompt information displayed on the display, thereby achieving intuitive, convenient and efficient collaborative interaction with the subject using a single display. During the collaborative interaction, the eye movement information, brain fnirs detection data and brain electrical data of the differentiated time periods are obtained.

[0049] In some other embodiments of the present application, during the performance of the task of each task block by the subject, the positive saccade and the negative saccade are pseudo-randomly and alternately performed in each task block at a ratio of 50±20% for the positive saccade and 50±20% for the negative saccade. Preferably, in each task block, the ratio of the positive saccade is 50%, and the ratio of the negative saccade is 50%. Controlling the ratio of the positive saccade and the ratio of the negative saccade within the range of 50%±20% can avoid excessive positive saccades or negative saccades to reduce the effect of stimulation on the subject, so that the brain electrical data and brain fnirs detection data that truly reflect the differences in brain function cannot be effectively obtained.

[0050] In some other embodiments of the present application, the ratio of the negative saccade is higher than the ratio of the positive saccade, such as performing the positive saccade and the negative saccade pseudo-randomly and alternately in each task block at a ratio of 30% for the positive saccade and 70% for the negative saccade. When the target object appears, the dominant response of the subject is to gaze at the movement of the target object, and the negative saccade is to suppress this dominant response. By increasing the ratio of the negative saccade, it is beneficial to better induce fnirs signal response and brain electrical signal response by mobilizing the control ability of the subject to suppress the dominant response, which indirectly reflects the control ability of the subject to suppress the dominant response, thereby facilitating more accurate brain function auxiliary assessment of the subject with cognitive dysfunction. In some other embodiments of the present application, the positive saccade includes a positive saccade look left response condition and a positive saccade look right response condition, and / or includes a positive saccade look up response condition and a positive saccade look down response condition; the negative saccade includes a negative saccade look left response condition and a negative saccade look right response condition, and / or includes a negative saccade look up response condition and a negative saccade look down response condition.

[0051] Specifically, the positive saccade look left response condition refers to that in the positive saccade, the second visual element presented in the third time period is at a left position on the screen, i.e. the subject is required to perform the action of gazing at the left of the second visual element after seeing the first mark presented on the screen.

[0052] Similarly, the positive saccade look to the right response condition, the positive saccade look to the upper side response condition, or the positive saccade look to the lower side response condition refers to the second visual element presented in the third time period in the positive saccade trial being located at the right, upper, or lower position of the screen, respectively. That is, the subject is required to follow the second visual element to the right, upper, or lower position after seeing the first mark presented on the screen.

[0053] The antisaccade look-to-left reaction condition refers to the situation in which, during the antisaccade trial, the second visual element presented in the third time period is located to the right of the screen. This means that after seeing the second logo presented on the screen, the subject is required to suppress their eyes and follow the second visual element, looking in the opposite direction to the left.

[0054] Similarly, the antisaccade look-to-the-right response condition, the antisaccade look-to-the-up response condition, or the antisaccade look-to-the-down response condition refers to the second visual element presented in the third time period of the antisaccade trial being located at the left, lower, or upper position of the screen, respectively. That is, the subject is required to suppress the eyes from following the second visual element and look in the opposite direction to the right, upper, or lower position after seeing the second mark presented on the screen.

[0055] In some other embodiments of the present application, in each task block, pseudo-random alternation is performed for the positive saccade look to the left reaction condition, the positive saccade look to the right reaction condition, the antisaccade look to the left reaction condition, and the antisaccade look to the right reaction condition, wherein the positive saccade look to the left reaction condition, the positive saccade look to the right reaction condition, the antisaccade look to the left reaction condition, and the antisaccade look to the right reaction condition are each performed at least once; and / or, in each task block, pseudo-random alternation is performed for the positive saccade look to the top reaction condition, the positive saccade look to the bottom reaction condition, the antisaccade look to the top reaction condition, and the antisaccade look to the bottom reaction condition, wherein the positive saccade look to the top reaction condition, the positive saccade look to the bottom reaction condition, the antisaccade look to the top reaction condition, and the antisaccade look to the bottom reaction condition are each performed at least once. In this way, it is possible to avoid the subject from having an expectation and practice effect on the gaze position when performing the tasks of positive saccade trials and antisaccade trials. It can truly reflect the subject's visual attention allocation in unexpected situations, thereby ensuring the accuracy of the collected eye movement information, brain fNIRS detection data and EEG data.

[0056] In the same task block, the forward and reverse saccade trials may include only trials in the left-right direction, only trials in the up-down direction, or trials in both the up-down and left-right directions. As a preferred embodiment, in the same task block, the forward and reverse saccade trials include only trials in the left-right direction, or only trials in the up-down direction, so that the task difficulty of each task block to be executed is moderate, making it easier to distinguish between normal subjects and subjects with mental illness based on the collected eye movement information, brain fNIRS detection data, and EEG data.

[0057] Furthermore, for each task block, the frequency of positive saccade trials and antisaccade trials is the same. Assuming a complete trimodal task consists of two task blocks, each task block includes a positive saccade look-to-left response condition, a positive saccade look-to-right response condition, an antisaccade look-to-left response condition, and an antisaccade look-to-right response condition. If, in the first task block, the positive saccade look-to-left response condition occurs 20% of the time, the positive saccade look-to-right response condition occurs 25% of the time, the antisaccade look-to-left response condition occurs 30% of the time, and the antisaccade look-to-right response condition occurs 25% of the time. Then, in the second task block, positive saccades to the left response condition occurred with a frequency of 20%, positive saccades to the right response condition occurred with a frequency of 25%, antisaccades to the left response condition occurred with a frequency of 30%, and antisaccades to the right response condition occurred with a frequency of 25%; or in the second task block, positive saccades to the right response condition occurred with a frequency of 20%, positive saccades to the left response condition occurred with a frequency of 25%, antisaccades to the right response condition occurred with a frequency of 30%, and antisaccades to the left response condition occurred with a frequency of 25%.

[0058] This allows the subject's brain to be activated at a relatively stable rhythm as they perform each task block, and the changes in blood oxygen levels in the relevant brain regions are also relatively stable, which helps reduce noise and interference in the collected fNIRS brain data. Furthermore, it reduces interference from background noise and improves the accuracy of the collected EEG data.

[0059] In some other embodiments of the present application, when a positive saccade trial in a task block is executed, the first visual element, the first identifier, and the second visual element of the positive saccade trial are presented in a preset screen line of sight constraint area; when an anti-saccade trial in a task block is executed, the first visual element, the second identifier, and the second visual element of the anti-saccade trial are presented in a preset screen line of sight constraint area; wherein, the preset screen line of sight constraint area is defined based on the 10%-40% position between the opposite sides of the screen and the 60%-90% position between the opposite sides of the screen, and the two side boundaries of the preset screen line of sight constraint area are respectively presented as target objects with obvious contrast to the screen color, and the second visual element is presented on one of the target objects.

[0060] Specifically, if Figure 2 As shown, the two sides of the preset screen sight constraint area are two white vertical solid lines. These two white vertical solid lines are the target objects that contrast clearly with the black screen. It can be seen that in each task block 201, the preset screen sight constraint area always exists to constrain the subject's gaze range.

[0061] Using a white vertical solid line as the target object is only an example and does not constitute a limitation on the target object of the preset screen line of sight constraint area. For example, the target object can also be a white dotted line, a red solid line, or a blue solid line, and there is no specific limitation on this.

[0062] like Figure 2 As shown, the white vertical solid line near the left side of the screen is at a position of 15% relative to the left side of the screen, and the white vertical solid line near the right side of the screen is at a position of 85% relative to the left side of the screen. This is only used as an example. The preset screen sight constraint area can be defined based on the 10%-40% position between the two opposite sides of the screen and the 60%-90% position between the two opposite sides of the screen. This can not only constrain the subject's sight line to the preset screen sight constraint area, but also avoid the sight line being concentrated in the center of the screen so that effective eye movement information cannot be collected. This is beneficial to controlling the subject's sight gaze position, reducing artifacts caused by free eye movements to the EEG data, and ensuring that the data is easy to analyze and compare.

[0063] For example, Figure 2 As shown, when the first visual element 204 is displayed, it is presented at the center of the preset screen sight constraint area; when the first mark 205 and the second mark 207 are displayed, they are presented at the center of the preset screen sight constraint area; when the second visual element 206 is displayed, it is presented on the target object on one side of the preset screen sight constraint area, preferably at the center of the target object. In this way, when the subject performs the task, the sight line is constrained to the preset screen sight constraint area, and during the third time period, the second visual element is presented on the target object, which is beneficial for controlling the gaze position of the subject's positive and negative saccades, improving the accuracy of the collected eye movement information, reducing the artifacts caused by free eye movements in the EEG data, and thus facilitating the collection of effective and accurate eye movement information and EEG data.

[0064] In some other embodiments of the present application, positive saccade trials and antisaccade trials are pseudo-randomly performed alternately in a preset screen sight constraint area, and the subject's eye movement information about the location of the second visual element within a preset third time period in each trial is collected. When the subject starts to perform the tasks in each task block, the preset screen sight constraint area will be presented on the screen to constrain the subject's sight within the preset screen sight constraint area. For example, Figure 2As described above, when the second visual element 206, i.e., a gray dot, is presented on the target object on one side of the preset screen sight constraint area during the preset third time period, the subject performs an eye movement action of looking at the position of the gray dot. At this time, an eye tracking device such as an eye tracker can be used to collect the subject's eye movement information during this period.

[0065] After collecting the eye movement information, the processor is used to obtain the subject's eye movement response to the second visual element according to the identification in the second time period based on the collected eye movement information, thereby obtaining the subject's behavioral data when performing the task.

[0066] like Figure 2 As shown, after the gray dots disappear during the third time period of positive saccade trial 202, the white dots from the first time period of antisaccade trial 203 begin to appear within the preset visual constraint area on the screen. In other words, during the execution of a task block using the display, the presentation of the first visual element from the first time period ends at the end of the third time period, allowing the subject to complete each trial sequentially based on the visual elements and symbols presented within the preset visual constraint area on the screen.

[0067] Alternatively, if the subject is performing the task of the last third time period in the last trial of a task block 201, then after the subject completes this task, that is, after the gray dots and target objects in the preset screen sight constraint area on the screen disappear, a cross pattern of the first time interval will appear on the screen, and the subject will then take a break for the first time interval. That is to say, after a task block is completed using the display, the third visual element of the first time interval will be presented at the end of the third time period. The third visual element has different appearance features from the first and second visual elements, so that the subject can easily know that a break is about to be taken based on the third visual element. Figure 2 The third visual element is a cross pattern, the first visual element is a white dot, and the second visual element is a gray dot. This is only used as an example and does not constitute a limitation to the specific solution.

[0068] Returning to the embodiment of the present application, in step S102, the subject's eye movement information within a preset third time period in each trial, the brain fNIRS detection data during the first time interval and in each task block, and the EEG data within a preset third time period from before the second time period in each trial to after the second time period, which is shorter than the first time period, are collected.

[0069] Herein, the steps of collecting the eye movement information, the brain fnirs detection data and the electroencephalogram data in S102 can be performed simultaneously with the task blocks in S101, that is, the data of the three modalities of the subject can be collected while the subject is provided with the task blocks by the display, so as to realize the data collection under the three-modality paradigm.

[0070] Specifically, the brain fnirs detection data in the first time interval and each task block represents the brain fnirs detection data in each task block and the first time interval adjacent to the task block. By collecting the brain fnirs detection data in each first time interval, the collected brain fnirs detection data can be used as a baseline in the analysis of the brain fnirs detection data in the adjacent period, so as to accurately analyze the change of the brain fnirs detection data of the subject in each task block.

[0071] Specifically, the electroencephalogram data collected in each trial is collected in a time period less than the first time period before the second time period and a preset third time period after the second time period, that is, the electroencephalogram data is collected no earlier than the first time period and no later than the third time period. In this way, the electroencephalogram data at a predetermined time after the start of each time period in each trial can be used as a baseline, so as to accurately analyze the change of the electroencephalogram data of the subject when responding to each trial. The predetermined time can be set according to actual experience, which can be 100ms, 200ms, and is not limited herein.

[0072] In some embodiments, the eye movement information of the subject when performing each trial task is collected, and the eye movement information at least includes a correctness representation parameter and an eye jump reaction time.

[0073] The correctness representation parameter may, for example, be the eye jump direction correctness, and the correctness representation parameter can reflect whether the subject can accurately focus or inhibit focusing on the target stimulus.

[0074] The eye jump reaction time is related to the processing speed of the subject to visual information, and the shorter the eye jump reaction time, the faster the concentration of attention, and the longer the eye jump reaction time, the subject may have certain problems in attention concentration.

[0075] Based on the embodiment of the application, the display is used to sequentially execute a preset number of task blocks at a preset first time interval, and the positive and negative saccade trials are alternately executed in each task block. The fnirs detection data, the EEG data and the eye movement information of the brain can be collected to obtain high-quality effective information. Specifically, for the fnirs detection data of the brain, sufficient effective and differentiated fnirs detection data of the brain can be collected; for the EEG data, the prediction and practice effect of the subject on the stimulation are effectively reduced, so that each trial stimulation can cause the maximum EEG response; for the eye movement information, the use demand of the brain function evaluation under different dimensions can be met, and the signal collection demand of the EEG device and the eye movement information collection demand can be met, and the high-quality collection of the eye movement-fnirs-EEG three-mode data of the subject during the execution of the above task blocks can be realized.

[0076] In various embodiments of the application, the subject can include a patient with a mental illness or a subject with a tendency of mental illness. The method for collecting eye movement information, brain fnirs detection data and EEG data provided by the embodiment of the application is especially suitable for evaluating the brain function condition of the patient with a mental illness or the subject with a tendency of mental illness, and is used for assisting the evaluation of the brain function of the patient with a mental illness or the subject with a tendency of mental illness to obtain the brain function evaluation result.

[0077] The brain function evaluation result is an objective biological index of the subject, and the brain function of the subject can be evaluated by combining the brain function evaluation result with other auxiliary indexes. For example, the brain function evaluation result can be a prediction value representing the normal degree of brain function, a concern level indicating whether the doctor needs to pay attention to it, and the like, which are not limited.

[0078] Specifically, the processor can obtain the brain function evaluation result of the subject based on the collected eye movement information, EEG data and brain fnirs detection data of the subject by using a rule-based data analysis method or a model-based data analysis method.

[0079] The rule-based data analysis method specifically includes: performing brain function evaluation according to the use rules of the eye movement information, the EEG data and the brain fnirs detection data prepared in advance.

[0080] The use rules include the use order of the data under different modalities, the threshold value representing the normal brain function of the subject, and the like, which are not limited and can be set by the user.

[0081] Exemplarily, the subject can be subjected to a first screening based on the electroencephalogram data and the brain fnirs detection data. In a case where the first screening result of the subject shows that the subject does not meet the preset electroencephalogram data normal standard and / or the preset brain fnirs detection data normal standard, the subject can be subjected to a second screening based on the eye movement information.

[0082] Further, the subject can be subjected to a first screening based on the brain fnirs detection data. In a case where the first screening result of the subject shows that the subject does not meet the preset brain fnirs detection data normal standard, the subject can be subjected to a second screening based on the electroencephalogram data. In a case where the first screening result and the second screening result show that the subject does not meet the preset electroencephalogram data normal standard and the preset brain fnirs detection data normal standard, the subject can be subjected to a third screening based on the eye movement information. In this way, the brain function evaluation result is obtained according to the screening results of each time.

[0083] Specifically, the specific method of obtaining the brain function evaluation result according to the screening results of each time includes: setting the brain function evaluation result of the subject who does not meet the preset eye movement information normal standard as a focus; setting the brain function evaluation result of the subject who meets the preset eye movement information normal standard, but does not meet the preset electroencephalogram data normal standard and / or the preset brain fnirs detection data normal standard as a general concern; and setting the brain function evaluation result of the subject who meets the preset eye movement information normal standard, and meets the preset electroencephalogram data normal standard and the preset brain fnirs detection data normal standard as no need to be concerned.

[0084] In this way, by pre-setting the use rules of the data under different modalities, the brain function condition of the subject can be preliminarily divided, so that the subsequent evaluation of the mental state of the subject is more targeted.

[0085] In some embodiments, the model-based data analysis method specifically includes: synchronously analyzing the data under three modalities of the subject based on the pre-trained model to obtain the brain function evaluation result.

[0086] Specifically, the label used in the process of training the model can include at least one of the evaluation results such as the subjective evaluation result of the doctor and the scale evaluation result of the examination. Based on a preset splicing order, the feature values corresponding to the three modalities are spliced to obtain the feature to be analyzed, so as to classify the feature to be analyzed by using the pre-trained model to obtain the brain function evaluation result of the subject. The three modalities are respectively an eye movement modality, an fnirs modality and an electroencephalogram modality.

[0087] In some embodiments, the characteristic value corresponding to the eye movement modality can be obtained by performing feature extraction on indicators such as accuracy characterization parameters, saccadic reaction time, and intra-individual variability of saccadic reaction time in the eye movement information; the characteristic value corresponding to the fNIRS modality can be obtained by performing feature extraction on the change in the brain fNIRS detection data in each task block compared with the brain fNIRS detection data during the first time interval; the characteristic value corresponding to the EEG modality can be obtained by performing feature extraction on the EEG data in each task block compared with the change in the EEG data at a predetermined time after the start of each time period in each trial, and can also be obtained by performing time domain analysis on the EEG data in each task block.

[0088] In this way, the subjects only need to perform one trimodal task paradigm to achieve accurate collection of eye movement information, brain fNIRS detection data, and EEG data, so as to accurately evaluate the brain function of the subjects, especially those with mental illness or those with mental illness tendencies. It also makes it easier for doctors to distinguish patients with mental disorders based on the objective biological indicators obtained from the evaluation, and to establish a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

[0089] In other embodiments of the present application, a processor can be used to obtain auxiliary brain function assessment results for subjects with schizophrenia, bipolar disorder, major depressive disorder, and ADHD based on the collected eye movement information, EEG data, and brain fNIRS detection data of the subject. For example, ADHD patients have difficulty controlling the generation of saccades and are more likely to make directional errors in antisaccade trials. Moreover, ADHD patients have greater activation in the dorsolateral prefrontal cortex than healthy controls. Because ADHD patients themselves have defects in executive functions such as attention and inhibitory control, when performing antisaccade tasks, they need to make more effort to suppress natural saccade reactions and make correct antisaccade movements. The enhanced activation of the dorsolateral prefrontal cortex may be a compensatory method adopted by the brain to make up for this deficiency in executive function.

[0090] Schizophrenia (SZ) patients exhibit larger P100 amplitudes in the left temporo-occipital region during antisaccades. This is likely due to deficits in inhibitory control, requiring the brain to work harder, activating more neurons (as evidenced by larger P100 amplitudes), to overcome this difficulty and successfully perform the antisaccade task. Patients with major depressive disorder (MDD) exhibit higher antisaccade error rates and reduced anticipatory activation in the dorsolateral prefrontal and inferior parietal cortices than healthy controls. This may be due to problems with the brain's neural regulatory mechanisms, making it difficult for MDD patients to effectively inhibit reflexive saccades. This reduced inhibitory ability may reflect deficits in cognitive control. Patients with bipolar disorder (BD) exhibit higher antisaccade error rates and longer reaction times than healthy controls, suggesting deficits in the cognitive control needed to inhibit automatic responses.

[0091] In this way, eye movement information combined with fNIRS brain data and EEG data can help doctors understand the subject's brain function status from multiple dimensions, providing more comprehensive information on brain functional activity, thereby facilitating the multi-dimensional assessment of the subject's brain function status and ultimately, accurately assessing the subject's brain function. At the same time, the acquired eye movement information, fNIRS brain data, and EEG data can collaboratively objectively and comprehensively reflect the impact of the subject's different brain function conditions on each modality of data, and can serve as effective data for auxiliary analysis of the subject's brain function status.

[0092] In some embodiments of the present application, a terminal is provided for providing a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data, such as Figure 3 As shown, the terminal 300 is configured to use the processor 301 to sequentially execute a preset number of task blocks for the subject at a preset first time interval, and pseudo-randomly alternately execute positive saccade trials and antisaccade trials in each task block. Executing positive saccade trials includes sequentially presenting a first visual element within a preset first time period, presenting a first identifier indicating positive saccades within a preset second time period, and presenting a second visual element within a preset third time period; executing antisaccade trials includes sequentially presenting a first visual element within a preset first time period, presenting a second identifier indicating antisaccades within a preset second time period, and presenting a second visual element within a preset third time period. The terminal 300 may also include a display 302 for sequentially displaying the various visual elements and marks on the screen of the display 302 in a preset order. The subject follows the various task prompts presented on the display 302 to perform the tasks of the preset number of task blocks.

[0093] Among them, the fNIRS detection data of the brain during the first time interval and in each task block, and the EEG data in a preset third time period from before the second time period in each trial to after the second time period, which is less than the length of the first time period, are used for collaborative analysis with the subject's eye movement information in the preset third time period in each trial to obtain the subject's auxiliary brain function assessment results.

[0094] Figure 4 A schematic diagram of a data acquisition system according to an embodiment of the present application is shown. The data acquisition system 400 includes an eye movement information acquisition device 401, an fNIRS device 402, and an EEG device 403, each of which is used to execute the collaborative interaction method for collecting eye movement information, fNIRS brain detection data, and EEG data as described in various embodiments of the present application.

[0095] The processor may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), or the like. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), or the like.

[0096] This application describes various operations or functions that can be implemented as software code or instructions or defined as software code or instructions. Such content can be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software code or instructions can be stored in a computer-readable storage medium and, when executed, can cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0097] The exemplary methods described herein can be at least partially machine or computer-implemented. In some embodiments, a computer-readable storage medium stores a computer program that, when executed, implements the collaborative interaction methods for collecting eye movement information, fNIRS brain data, and EEG data as described in various embodiments of the present invention.

[0098] In some embodiments, a computer readable storage medium stores a computer program, which when executed via a processor provides a three-modal paradigm of eye movement information, brain fnirs detection data, and electroencephalogram data, specifically comprising for a subject, sequentially performing a preset number of task blocks at a preset first time interval, in each of which a positive saccade trial and a negative saccade trial are pseudo-randomly alternately performed, performing the positive saccade trial comprising sequentially presenting a first visual element for a preset first time period, presenting a first indicator indicating a positive saccade for a preset second time period, and presenting a second visual element for a preset third time period; and performing the negative saccade trial comprising sequentially presenting the first visual element for the preset first time period, presenting a second indicator indicating a negative saccade for the preset second time period, and presenting the second visual element for the preset third time period.

[0099] Implementations of such methods can include software code, e.g., microcode, assembly language code, a higher-level languages code, etc. Various programs or program modules can be created using a variety of software programming techniques. For example, program portions or program modules can be designed in or by Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules can be integrated into a computer system and / or computer readable medium. Such software code can include computer readable instructions for performing various methods. The software code can form part of the computer program product or computer program module. Moreover, in examples, the software code can be tangibly stored on one or more volatile, non-transitory, or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media can include, but are not limited to, hard disks, removable magnetic disks, removable optical disks, magnetic cassettes, memory cards or sticks, random access memories (RAMs), read only memories (ROMs), and the like.

[0100] Furthermore, although example embodiments have been described herein, the scope of their protection is to be understood as including any and all equivalents based on the following claims, modifications, permutations, combinations, sub-combinations, and / or applications for which the spirit or principles of the application are employed. The claims are to be interpreted broadly based on the language employed in the claims and not limited to examples described in the specification, which examples are to be interpreted as non-exhaustive. Thus, the present specification and examples are to be considered exemplary only, with the true scope and spirit of the application indicated by the following claims and their full scope of equivalents.

[0101] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of their embodiments) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above detailed description, various features may be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that is not claimed for protection is essential to any claim. On the contrary, the subject matter of the present application may have less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein into the detailed description as examples or embodiments, with each claim independently serving as a separate embodiment, and it is contemplated that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the appended claims and the full scope of equivalents to which such claims are entitled.

[0102] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A collaborative interactive method for collecting eye movement information, brain fNIRS detection data, and EEG data, characterized in that: The collaborative interaction method is applicable to subjects suffering from mental illness or having a tendency to mental illness, and includes: Using a display, the subject sequentially performs a preset number of task blocks at a preset first time interval, and pseudo-randomly alternately performs positive saccade trials and antisaccade trials in each task block, wherein performing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first identifier indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; performing the antisaccade trial includes sequentially presenting the first visual element within the preset first time period, presenting a second identifier indicating an antisaccade within the preset second time period, and presenting the second visual element within the preset third time period, wherein the first identifier and the second identifier have different appearance characteristics, and the appearance characteristics are one or more of color characteristics, shape characteristics, and size characteristics; The eye movement information of the subjects in the preset third time period of each trial, the brain fNIRS detection data during the first time interval and in each task block, and the EEG data in the preset third time period before the second time period of each trial (which is shorter than the first time period) and after the second time period are collected to establish a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

2. The collaborative interaction method according to claim 1, characterized in that: For a complete task, 2-8 task blocks are sequentially executed at a first time interval of 10s-60s, and the duration of each task block is 10s-60s.

3. The collaborative interaction method according to claim 2, characterized in that: In each task block, the positive saccade trials and the antisaccade trials were pseudo-randomly alternating with a ratio of 50±20% for the positive saccade trials and a ratio of 50±20% for the antisaccade trials.

4. The collaborative interaction method according to claim 3, characterized in that: In the positive saccade trial, presenting the first identifier to instruct the subject to perform an action of gazing at the location of the second visual element within a preset third time period; In the antisaccade trial, presenting the second identifier to instruct the subject to perform an action of gazing at a position equidistant from a position of the second visual element within a preset third time period; The positive saccade trial includes a positive saccade to the left reaction condition and a positive saccade to the right reaction condition, and / or includes a positive saccade to the top reaction condition and a positive saccade to the bottom reaction condition; The antisaccade trial includes an antisaccade looking to the left reaction condition and an antisaccade looking to the right reaction condition, and / or includes an antisaccade looking to the top reaction condition and an antisaccade looking to the bottom reaction condition.

5. The collaborative interaction method according to claim 4, characterized in that: In each task block, pseudo-randomly alternately execute the positive saccade look-to-left response condition, the positive saccade look-to-right response condition, the antisaccade look-to-left response condition, and the antisaccade look-to-right response condition. Each of the positive saccade look-to-left response condition, the positive saccade look-to-right response condition, the antisaccade look-to-left response condition, and the antisaccade look-to-right response condition is executed at least once. And / or, in each task block, the positive saccade looking up reaction condition, the positive saccade looking down reaction condition, the anti-saccade looking up reaction condition and the anti-saccade looking down reaction condition are pseudo-randomly alternatingly executed, wherein the positive saccade looking up reaction condition, the positive saccade looking down reaction condition, the anti-saccade looking up reaction condition and the anti-saccade looking down reaction condition are each executed at least once.

6. The collaborative interaction method according to any one of claims 1 to 5, characterized in that: In each trial, the first time period is 200ms-2000ms, the second time period is 500ms-2000ms, and the third time period is 1000ms-2500ms.

7. The collaborative interaction method according to claim 6, characterized in that: When executing a positive saccade trial in the task block, the first visual element, the first identifier, and the second visual element of the positive saccade trial are presented in a preset screen vision constraint area; when executing an antisaccade trial in the task block, the first visual element, the second identifier, and the second visual element of the antisaccade trial are presented in a preset screen vision constraint area; wherein the preset screen vision constraint area is defined based on the 10%-40% position between the opposite sides of the screen and the 60%-90% position between the opposite sides of the screen, and the two side boundaries of the preset screen vision constraint area are respectively presented as target objects with obvious contrast with the screen color, and the second visual element is presented on one of the target objects.

8. The collaborative interaction method according to claim 6, characterized in that: The first visual element and the second visual element have different appearance characteristics.

9. The collaborative interaction method according to claim 7, characterized in that: pseudo-randomly alternating positive saccade trials and antisaccade trials in the preset visual constraint area of ​​the screen, collecting eye movement information of the subject regarding the location of the second visual element within a preset third time period in each trial, and obtaining, using a processor, an eye movement response result of the subject regarding the second visual element according to the identifier within the second time period based on the eye movement information; During execution of a task block using the display, jumping to the presentation of the first visual element of the first time period at the end of the third time period; Alternatively, after a task block is executed using the display, the display jumps to the third visual element presentation of the first time interval at the end of the third time period, wherein the third visual element has different appearance features from the first visual element and the second visual element.

10. The collaborative interaction method according to any one of claims 1 to 5, characterized in that: The eye movement information at least includes accuracy characterization parameters and saccade reaction time.

11. The collaborative interaction method according to any one of claims 1 to 5, characterized in that: Before executing the first task block, a second time interval is executed, and the duration of the second time interval is 10s-60s.

12. A terminal for providing a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data, characterized in that: The terminal is configured to utilize a processor to: A subject suffering from a mental illness or having a tendency toward a mental illness is administered a preset number of task blocks sequentially at a preset first time interval, pseudo-randomly alternating positive saccade trials and antisaccade trials in each task block, wherein the positive saccade trials include sequentially presenting a first visual element within a preset first time period, presenting a first marker indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; Executing the antisaccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a second identifier indicating an antisaccade within a preset second time period, and presenting a second visual element within a preset third time period, wherein the first identifier and the second identifier have different appearance characteristics, and the appearance characteristics are one or more of color characteristics, shape characteristics, and size characteristics; Among them, the fNIRS detection data of the brain during the first time interval and in each task block, and the EEG data in a preset third time period from before the second time period in each trial to after the second time period, which is less than the length of the first time period, are used for collaborative analysis with the eye movement information of the subject in the preset third time period in each trial to obtain the auxiliary assessment results of the subject's brain function, so as to establish a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

13. A data acquisition system, characterized in that: The data acquisition system includes an eye movement information acquisition device, an fNIRS device, and an electroencephalogram (EEG) device, each of which is used to execute the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data as described in any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed, provides a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data via a processor, specifically including: For subjects suffering from mental illness or with a tendency to mental illness, a preset number of task blocks are performed in sequence at a preset first time interval, and positive saccade trials and antisaccade trials are performed pseudo-randomly and alternately in each task block, wherein performing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first identifier indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; performing the antisaccade trial includes sequentially presenting the first visual element within the preset first time period, presenting a second identifier indicating an antisaccade within the preset second time period, and presenting the second visual element within the preset third time period, wherein the first identifier and the second identifier have different appearance characteristics, and the appearance characteristics are one or more of color characteristics, shape characteristics, and size characteristics.

Images