Cooperative interaction method, terminal and system for collecting eye movement information, brain fnirs detection data and electroencephalogram data

Abstract

The invention provides a collaborative interaction method, terminal and system for collecting eye movement information, brain fnirs detection data and electroencephalogram data, and the method comprises the steps: sequentially executing a preset number of task blocks for a subject at a preset first time interval by using a display, performing positive eye jump trials and negative eye jump trials alternately in a pseudo-random manner in each task block, wherein the execution of the positive eye jump trials includes sequentially presenting a first visual element, a first identifier indicating positive eye jump and a second visual element; the step of executing the anti-eye-jump test includes sequentially presenting the first visual element, a second identifier for indicating anti-eye-jump and the second visual element; the method comprises the following steps: collecting eye movement information, brain fnirs detection data and electroencephalogram data of a subject in each test time; in this way, the obtained eye movement information, the brain fnirs detection data and the electroencephalogram data can be cooperatively used for auxiliary assessment of brain functions related to cognitive impairment, and a more accurate classification model can be conveniently established for subsequent diagnosis of cognitive impairment.

Description

Technical Field

[0001] The present application relates to the technical field of brain function auxiliary assessment, and specifically to a collaborative interaction method, terminal and system for collecting eye movement information, brain fNIRS detection data and electroencephalogram data. Background Art

[0002] Cognitive dysfunction, including schizophrenia (SZ), bipolar disorder (BD), major depressive disorder (MDD) and attention deficit hyperactivity disorder (ADHD), presents diverse and complex clinical symptoms. When distinguishing these symptoms, doctors often need to rely on their personal clinical experience. As the number of patients continues to increase, the pressure on doctors in diagnosis also increases. In order to alleviate this problem, researchers have developed a variety of brain function auxiliary assessment methods to assist doctors in improving the speed of diagnosis.

[0003] Among the related technologies, functional near-infrared spectroscopy (fNIRS) has good spatial resolution, electroencephalogram (EEG) technology has excellent temporal resolution, and eye tracking technology can provide important information about visual attention and cognitive processing. Although using a single technology to assist in the assessment of brain function is helpful in distinguishing the types of cognitive dysfunction, the clinical symptoms of different cognitive dysfunctions may be similar when evaluated using a single technology, making it impossible to accurately distinguish different types of cognitive dysfunction.

[0004] The blood oxygen signal collected by fNIRS is a slow signal, the EEG signal collected by EEG is a fast signal, and the eye movement information collected by eye tracking technology is a fast-changing behavioral signal. Therefore, the method suitable for fNIRS signal collection is usually not suitable for EEG signal collection and eye movement information collection. If you want to obtain accurate evaluation indicators in different dimensions, you need to conduct multiple brain function auxiliary evaluations, which brings inconvenience to the examinee. Therefore, there is still a lot of room for improvement in the technology that combines data from multiple modalities to perform brain function auxiliary evaluation. Summary of the invention

[0005] This application is proposed in view of the above technical problems existing in the prior art. This application aims to provide a collaborative interaction method for collecting eye movement information, brain fNIRS detection data and EEG data, which can provide a trimodal task paradigm suitable for simultaneously collecting effective eye movement information, brain fNIRS detection data and EEG data, and can realize intuitive, convenient and efficient collaborative interaction with the subject using a single display. In the process of collaborative interaction, eye movement information, brain fNIRS detection data and EEG data of different time periods are obtained. The acquired eye movement information, brain fNIRS detection data and EEG data can be coordinated to objectively and comprehensively reflect the influence of different brain function conditions of the subject on each modal data, so as to facilitate the establishment of a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

[0006] According to a first scheme of the present application, a collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and electroencephalogram data is provided. The collaborative interaction method includes using a display to sequentially execute a preset number of task blocks for a subject at a preset first time interval, and pseudo-randomly and alternately execute positive saccade trials and anti-saccade trials in each task block, wherein executing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing the anti-saccade trial includes sequentially presenting the first visual element within a preset first time period, presenting a second mark indicating an anti-saccade within a preset second time period, and presenting the second visual element within a preset third time period; and collecting the subject's eye movement information within a preset third time period in each trial, the brain fNIRS detection data during the first time interval and in each task block, and the electroencephalogram data within a preset third time period from before the second time period to after the second time period in each trial, which is less than the length of the first time period.

[0007] According to a second scheme of the present application, a terminal for providing a trimodal paradigm of eye movement information, brain fNIRS detection data, and electroencephalogram data is provided, wherein the terminal is configured to use a processor to: sequentially execute a preset number of task blocks for a subject at a preset first time interval, and pseudo-randomly and alternately execute positive saccade trials and anti-saccade trials in each task block, wherein executing the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing the anti-saccade trial includes sequentially presenting the first visual element within a preset first time period, presenting a second mark indicating an anti-saccade within a preset second time period, and presenting the second visual element within a preset third time period; wherein the brain fNIRS detection data during the first time interval and in each task block, and the electroencephalogram data within a preset third time period from a time length less than the first time period before the second time period in each trial to a time length less than the first time period after the second time period, are used to coordinately analyze with the eye movement information of the subject within the preset third time period in each trial to obtain an auxiliary assessment result of the subject's brain function.

[0008] According to the third scheme of the present application, a data acquisition system is provided, which includes an eye movement information acquisition device, an fNIRS device, and an EEG device, each of which is used to execute the collaborative interaction method for collecting eye movement information, brain fNIRS detection data and EEG data as described in the various embodiments of the present application.

[0009] According to the fourth scheme of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data as described in the various embodiments of the present application is implemented.

[0010] According to a fifth scheme of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, a processor is used to provide a trimodal paradigm of eye movement information, brain fNIRS detection data, and EEG data, specifically comprising executing a preset number of task blocks in sequence for a subject at a preset first time interval, and pseudo-randomly alternatingly executing positive saccade trials and anti-saccade trials in each task block, wherein executing the positive saccade trial comprises sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing the anti-saccade trial comprises sequentially presenting the first visual element within the preset first time period, presenting a second mark indicating an anti-saccade within the preset second time period, and presenting the second visual element within the preset third time period.

[0011] Compared with the prior art, the beneficial effects of the embodiments of the present application are: The collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data provided in the embodiment of the present application uses a display to sequentially execute a preset number of task blocks for the subject at a preset first time interval, pseudo-randomly and alternately execute positive saccade trials and anti-saccade trials in each task block, present a first visual element in a first time period in each trial, present an indicator mark in a second time period, and present a second visual element in a third time period; in this way, for the brain fNIRS detection data, since the task block contains positive saccade trials and anti-saccade trials, it is sufficient to collect effective and differentiated brain fNIRS detection data, and each task block The design of the time interval meets the time requirements for the acquisition and analysis of fNIRS signals, so that high-quality brain fNIRS detection data can be obtained; for EEG data, since positive saccade trials and anti-saccade trials appear pseudo-randomly and alternately, the prediction and practice effects of the subjects on the stimulation are effectively reduced, so that each trial stimulation can cause the maximum EEG response; for eye movement information, different types of trials can meet the use requirements of brain function evaluation in different dimensions and meet the signal acquisition requirements and eye movement information acquisition requirements of EEG equipment. The eye movement information obtained by executing positive saccade trials and anti-saccade trials can effectively evaluate and verify the brain function related to cognitive dysfunction. Therefore, when the subjects perform the above task blocks, they can simultaneously collect high-quality eye movement-fNIRS-EEG trimodal data, and the data collected in each modality can be used for auxiliary evaluation of brain function related to cognitive dysfunction.

[0012] In this way, based on the collaborative interaction method provided by the present application, a trimodal task paradigm can be provided to the subject, so that the subject can simultaneously obtain effective and accurate eye movement information, brain fNIRS detection data and EEG data while executing the trimodal task paradigm provided by the embodiment of the present application.

[0013] This allows for an accurate assessment of the brain function of the subjects, especially those with mental illness or a tendency to mental illness. It also makes it easier for doctors to differentiate between patients with mental disorders based on the objective biological indicators obtained from the assessment, and to establish a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

[0014] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. Similar reference numerals with letter suffixes or different letter suffixes may represent different examples of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not by way of limitation, and together with the description and claims, are used to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary and are not intended to be exhaustive or exclusive embodiments of the present method, apparatus, system, or non-transitory computer-readable medium having instructions for implementing the method.

[0016] Figure 1 A flow chart of a collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data provided according to an embodiment of the present application is shown.

[0017] Figure 2 A schematic diagram of displaying various task blocks based on a display according to an embodiment of the present application is shown.

[0018] Figure 3 A schematic diagram of a terminal for providing a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data according to an embodiment of the present application is shown.

[0019] Figure 4 A schematic diagram of a data acquisition system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present application.

[0021] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0022] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined here. Techniques and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the techniques and equipment should be considered as part of the specification.

[0023] Figure 1 A flowchart of a collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data provided in accordance with an embodiment of the present application is shown. Specifically, the method shown in step S101-step S102 is executed. In this embodiment, the arrows shown in the figure for each step are only examples of the execution order, not limitations. The technical solution of the present application is not limited to the execution order described in the embodiment. The various steps in the execution order can be combined, decomposed, or exchanged, as long as the logical relationship of the execution content is not affected.

[0024] For example, before the subject performs the trimodal task paradigm, the subject can be equipped with an eye tracker suitable for collecting eye movement information and a head cap for collecting fNIRS detection data and EEG data of the brain. The installation position of the fNIRS signal acquisition probe and the installation position of the EEG electrode configured in the head cap can be set by the user according to prior medical knowledge, and no specific limitation is made to this. Among them, the light source in the eye tracker will not interfere with the collection of fNIRS detection data and EEG data of the brain.

[0025] After the subject wears the eye tracker for collecting eye movement information and the head cap for collecting brain fNIRS detection data and EEG data, the trimodal task can be started to simultaneously collect the subject's eye movement information, brain fNIRS detection data and EEG data while performing the trimodal task.

[0026] Specifically, in step S101, a display is used to sequentially execute a preset number of task blocks for the subject at a preset first time interval, and positive saccade trials and anti-saccade trials are pseudo-randomly and alternately executed in each task block, wherein the execution of the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; the execution of the anti-saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a second mark indicating an anti-saccade within a preset second time period, and presenting a second visual element within a preset third time period.

[0027] Specifically, a task prompt requiring the subject to perform a trimodal task can be displayed on the screen of the display, so that the subject performs the corresponding trimodal task according to the task prompt displayed on the display, thereby simultaneously collecting the subject's eye movement information, brain fNIRS detection data, and EEG data when performing the trimodal task. Based on the collaborative interaction method provided in the embodiment of the present application, the acquired eye movement information, brain fNIRS detection data, and EEG data are more accurate and reliable, and can objectively and comprehensively reflect the impact of the subject's different brain function conditions on each modal data, and can be used as effective data combined with auxiliary analysis of the subject's brain function condition.

[0028] like Figure 2 As shown, before presenting the first task block 201, a cross symbol is presented on the screen to prompt the subject to take a rest before executing the first task block 201, so that the subject knows that the task is about to start and is mentally prepared to perform the task. That is to say, in some other embodiments, before executing the first task block, a second time interval is performed, and the duration of the second time interval is 10s-60s, which is conducive to allowing the subject to adjust his state by taking a rest during the second time interval and concentrate on performing subsequent tasks.

[0029] After the subject completes the task of task block 201, he / she rests for a first time interval, then continues to perform the task of task block 201, rests for a first time interval again, and then continues to perform the task of task block 201, and so on, until the subject completes the preset number of task blocks 201. For example, the first time interval may be 20s, and the second time interval may also be 20s, that is, Figure 2 The first time interval shown may be the same as the second time interval. In addition, after the subject completes each task block 201, the subject will take a rest according to the first time interval, thereby reducing the task load of the subject.

[0030] Figure 2 Only the first task block 201 and the first time interval adjacent to the first task block 201 are shown, and the remaining task blocks 201 and the first time intervals adjacent to the first task block 201 are indicated by ellipsis, and, Figure 2 The content shown is only for facilitating the understanding of the technical solution and does not constitute a limitation on the specific solution.

[0031] The first time interval may also be other time periods such as 30s, 40s, etc. The preset number of task blocks 201 is at least 2, which can be specifically set by doctors or relevant researchers according to medical experience.

[0032] In a preferred embodiment of the present application, for a complete task, with a first time interval of 10s-60s, 2-8 task blocks are sequentially executed, and the duration of each task block is 10s-60s. And the duration of each task block is consistent. The first time interval is set to 10s-60s, which is conducive to the recovery of blood oxygen level, while alleviating the task load of the subject, and the overall task duration will not be too long due to the long rest time interval. And, 2-8 task blocks are sequentially executed, and the duration of each task block is 10s-60s, which can cause a good fNIRS signal response, can meet the extraction and analysis requirements of collecting brain fNIRS detection data, and obtain accurate and effective brain fNIRS detection data.

[0033] like Figure 2 , pseudo-randomly performing positive saccade trials 202 and anti-saccade trials 203 alternately in each task block 201, assuming Figure 2 A task block 201 shown includes two positive saccade trials 202 and three anti-saccade trials 203. The subject may first perform a positive saccade trial 202, then perform an anti-saccade trial 203, then perform an anti-saccade trial 203, then perform a positive saccade trial 202, and then perform an anti-saccade trial 203. Alternatively, the subject may first perform an anti-saccade trial 203, then perform a positive saccade trial 202, then perform an anti-saccade trial 203, then perform an anti-saccade trial 203, and then perform a positive saccade trial 202.

[0034] That is to say, the order of alternating presentation of positive saccade trials and anti-saccade trials in each task block can be random, which is helpful to avoid the subjects' anticipation of the subsequent saccade movements.

[0035] In this embodiment, by pseudo-randomly alternating presentation of positive saccade trials and anti-saccade trials and repeatedly executing each task block, the fNIRS signal response of the brain can be better induced, and the EEG response that meets the use requirements can be collected.

[0036] This is only an illustrative description and does not constitute a limitation on the specific solution. Figure 2 As shown, the background color of each screen displayed is black, that is, Figure 2 A black screen is used as an example for explanation.

[0037] like Figure 2, a white dot at the center of a black screen, a five-pointed star pattern at the center of a black screen, and a gray dot at the edge of a black screen can be sequentially displayed in a positive saccade trial 202. The three screens are displayed in sequence to complete a positive saccade trial 202. Among them, the task block 201 only shows one positive saccade trial 202, and the other positive saccade trials 202 are all represented by ellipsis.

[0038] In the positive saccade trial 202, the first visual element 204 is a white dot at the center of the black screen, the first logo 205 is a five-pointed star pattern at the center of the black screen, and the second visual element 206 is a gray dot at the edge of the black screen. In addition, in the positive saccade trial 202, the first visual element 204 is presented in the first time period, the first logo 205 is presented in the second time period, and the second visual element 206 is presented in the third time period.

[0039] In the antisaccade trial 203, a white dot at the center of a black screen, a heart-shaped pattern at the center of a black screen, and a gray dot at the edge of a black screen can be sequentially displayed. The three screens are displayed sequentially to complete an antisaccade trial 203. Among them, the task block 201 only shows one antisaccade trial 203, and the other antisaccade trials 203 are all represented by ellipsis.

[0040] In the antisaccade trial 203, the first visual element 204 is a white dot at the center of the black screen, the second mark 207 is a heart-shaped pattern at the center of the black screen, and the second visual element 206 is a gray dot at the edge of the black screen. In the antisaccade trial 203, the first visual element 204 is presented in the first time period, the second mark 207 is presented in the second time period, and the second visual element 206 is presented in the third time period.

[0041] Then, in some other embodiments, in each trial, the first time period is 200ms-2000ms, the second time period is 500ms-2000ms, and the third time period is 1000ms-2500ms. Exemplarily, Figure 2 The first time period shown in the figure can be 0.2s, 0.5s, 0.8s, 1s, 1.3s, 1.5s or 1.7s, the second time period can be 0.5s, 0.9s, 1.3s, 1.5s, 1.7s or 1.9s, and the third time period can be 1s, 1.3s, 1.7s, 2s, 2.2s or 2.4s, that is, the subject performs a positive saccade trial 202 or performs an anti-saccade trial 203, and the duration is not less than 1.7s, which is moderate for the subject. In this way, the difficulty of the task performed by the subject can be controlled so that the subject can complete the entire trimodal task well.

[0042] In this embodiment, the first visual element, the indicator mark and the second visual element are presented in different time periods and on different screens. By reasonably setting different time periods for executing tasks, especially the third time period being longer, free eye movements can be effectively controlled, the overlap of brain reactions caused by stimulation tasks can be reduced, and interference such as artifacts related to eye movements can be reduced, which is conducive to collecting accurate and effective eye movement information and EEG data.

[0043] In some other embodiments of the present application, the first visual element and the second visual element have different appearance features, and the first logo and the second logo have different appearance features. The appearance features may be one or more of color features, shape features, and size features, and are not limited thereto, as long as the first visual element and the second visual element can be distinguished by the appearance features, and the first logo and the second logo can be distinguished by the appearance features. Figure 2 The first mark 205 and the second mark 207 have different shape features, and the first visual element 204 and the second visual element 206 have different color features.

[0044] In some other embodiments of the present application, in the positive saccade trial, the first mark is presented to instruct the subject to perform an action of gazing at the location of the second visual element within a preset third time period. Figure 2 In the positive saccade trial 202, when the five-pointed star pattern is presented on the black screen in the second time period, it means that the subject should fixate on the position of the next gray dot on the black screen in the third time period after the five-pointed star pattern disappears, that is, the five-pointed star pattern is used to instruct the subject to perform the action of fixing the position of the gray dot in the next time period.

[0045] In the antisaccade trial 203, when the heart-shaped pattern is presented on the black screen in the second time period, it means that the subject should gaze at a position equidistant from the position of the gray dot on the black screen in the third time period after the heart-shaped pattern disappears, that is, the heart-shaped pattern is used to instruct the subject to perform an action of gazing at a position equidistant from the position of the gray dot in the next time period. In other words, in the antisaccade trial, the second mark is presented to instruct the subject to perform an action of gazing at a position equidistant from the position of the second visual element in the preset third time period.

[0046] In addition, the first and second identifiers may be set based on relevant information such as the subject's age or mental condition. For example, if the subject is a child, the first and second identifiers may be set as cartoon symbols to make the child subject interested in the task to be performed.

[0047] Among them, before the subject formally starts to perform the trimodal task paradigm, the subject can be asked to perform a practice task to let the subject know that when seeing the first mark, the subject should perform the action of fixating the position of the second visual element, and when seeing the second mark, the subject should perform the action of fixating the position equidistant from the position of the second visual element. In this way, the subject can perform the tasks of each task block based on the task prompt information displayed on the display, so that a single display can be used to achieve intuitive, convenient and efficient collaborative interaction with the subject, and in the process of collaborative interaction, eye movement information, brain fNIRS detection data and EEG data that distinguish time periods can be obtained.

[0048] In some other embodiments of the present application, when the subject performs the tasks of each task block, in each task block, the positive saccade trial and the anti-saccade trial are pseudo-randomly alternately performed with the ratio of the positive saccade trial being 50±20% and the ratio of the anti-saccade trial being 50±20%. Preferably, in each task block, the ratio of the positive saccade trial is 50%, and the ratio of the anti-saccade trial is 50%. The ratio of the positive saccade trial and the ratio of the anti-saccade trial are controlled to be adjusted within the range of 50%±20%, which can avoid too many positive saccade trials or anti-saccade trials from reducing the effect of stimulation to the subject, thereby being unable to effectively obtain EEG data and brain fNIRS detection data that truly reflect the difference in brain function status.

[0049] Among them, the ratio of anti-saccade trials is allowed to be higher than the ratio of positive saccade trials. For example, in each task block, the positive saccade trials and the anti-saccade trials are pseudo-randomly performed alternately with the ratio of positive saccade trials being 30% and the ratio of anti-saccade trials being 70%. When the target object appears, the subject's dominant reaction is to look at the movement of the target object, while the anti-saccade trial is to inhibit this dominant reaction. By increasing the ratio of anti-saccade trials, it is beneficial to mobilize the subject's control ability to inhibit the dominant reaction to better induce fNIRS signal response and EEG signal response, which indirectly reflects the subject's control ability to inhibit the dominant reaction, thereby facilitating a more accurate auxiliary assessment of brain function for subjects with cognitive dysfunction. In some other embodiments of the present application, the positive saccade trial includes a positive saccade looking to the left reaction condition and a positive saccade looking to the right reaction condition, and / or includes a positive saccade looking up reaction condition and a positive saccade looking down reaction condition; the anti-saccade trial includes an anti-saccade looking to the left reaction condition and an anti-saccade looking to the right reaction condition, and / or includes an anti-saccade looking up reaction condition and an anti-saccade looking down reaction condition.

[0050] Specifically, the positive saccade to the left reaction condition refers to that in the positive saccade trial, the second visual element presented in the third time period is located at the left side of the screen, that is, the subject is required to follow the second visual element to the left after seeing the first mark presented on the screen.

[0051] Similarly, the positive saccade to the right response condition, the positive saccade to the upper response condition or the positive saccade to the lower response condition respectively refer to the second visual element presented in the third time period in the positive saccade trial being located at the right, upper or lower part of the screen respectively, that is, the subject is required to follow the second visual element to the right, upper or lower part of the screen after seeing the first mark presented on the screen.

[0052] The antisaccade look-to-left reaction condition means that in the antisaccade trial, the second visual element presented in the third time period is located at the right side of the screen, that is, the subject is required to suppress the eyes from following the second visual element and look in the opposite direction to the left after seeing the second mark presented on the screen.

[0053] Similarly, the antisaccade look to the right reaction condition, the antisaccade look up reaction condition or the antisaccade look down reaction condition respectively refers to the second visual element presented in the third time period in the antisaccade trial being located at the left, lower or upper part of the screen respectively, that is, the subjects are required to suppress their eyes from following the second visual element and look in the opposite direction to the right, upper or lower part after seeing the second mark presented on the screen.

[0054] In some other embodiments of the present application, in each task block, pseudo-randomly and alternately execute the positive eye saccade look left reaction condition, the positive eye saccade look right reaction condition, the anti-eye saccade look left reaction condition and the anti-eye saccade look right reaction condition, wherein the positive eye saccade look left reaction condition, the positive eye saccade look right reaction condition, the anti-eye saccade look left reaction condition and the anti-eye saccade look right reaction condition are each executed at least once; and / or, in each task block, pseudo-randomly and alternately execute the positive eye saccade look up reaction condition, the positive eye saccade look down reaction condition, the anti-eye saccade look up reaction condition and the anti-eye saccade look down reaction condition, wherein the positive eye saccade look up reaction condition, the positive eye saccade look down reaction condition, the anti-eye saccade look up reaction condition and the anti-eye saccade look down reaction condition are each executed at least once. In this way, it is possible to avoid the subject from having an expectation and practice effect on the gaze position when performing the tasks of positive eye saccade trials and anti-eye saccade trials. It can truly reflect the subject's visual attention allocation in unexpected situations, thereby ensuring the accuracy of the collected eye movement information, brain fNIRS detection data and EEG data.

[0055] Among them, in the same task block, the positive and negative saccade trials may only include trials in the left and right direction, or only include trials in the up and down direction, or may also include trials in the up and down direction and the left and right directions. As a preferred embodiment, in the same task block, the positive and negative saccade trials only include trials in the left and right direction, or only include trials in the up and down direction, so that the task difficulty of each task block to be executed is moderate, so that normal subjects and mental illness subjects can be distinguished based on the collected eye movement information, brain fNIRS detection data and EEG data.

[0056] In addition, for each task block, the frequency of executing positive saccade trials and antisaccade trials is the same. Assuming that a complete trimodal task includes two task blocks, each task block includes positive saccade look left response condition, positive saccade look right response condition, antisaccade look left response condition and antisaccade look right response condition, if the frequency of positive saccade look left response condition in the first task block is 20%, the frequency of positive saccade look right response condition is 25%, the frequency of antisaccade look left response condition is 30%, and the frequency of antisaccade look right response condition is 25%. Then, in the second task block, positive saccades to the left response condition occurred with a frequency of 20%, positive saccades to the right response condition occurred with a frequency of 25%, antisaccades to the left response condition occurred with a frequency of 30%, and antisaccades to the right response condition occurred with a frequency of 25%; or in the second task block, positive saccades to the right response condition occurred with a frequency of 20%, positive saccades to the left response condition occurred with a frequency of 25%, antisaccades to the right response condition occurred with a frequency of 30%, and antisaccades to the left response condition occurred with a frequency of 25%.

[0057] In this way, when the subject performs tasks in each task block, the brain can be activated at a relatively stable rhythm, and the changes in the blood oxygen content in the relevant brain areas will also be relatively stable, which is conducive to reducing the noise and interference in the collected brain fNIRS detection data. In addition, it can also reduce the interference of background noise and improve the accuracy of the collected EEG data.

[0058] In some other embodiments of the present application, when a positive saccade trial in a task block is executed, a first visual element, a first identifier, and a second visual element of the positive saccade trial are presented in a preset screen line of sight constraint area; when an anti-saccade trial in the task block is executed, a first visual element, a second identifier, and a second visual element of the anti-saccade trial are presented in a preset screen line of sight constraint area; wherein the preset screen line of sight constraint area is defined based on a 10%-40% position between two opposite sides of the screen and a 60%-90% position between two opposite sides of the screen, and two side boundaries of the preset screen line of sight constraint area are respectively presented as target objects with obvious contrast with the screen color, and the second visual element is presented on one of the target objects.

[0059] Specifically, Figure 2 As shown, the two sides of the preset screen sight constraint area are two white vertical solid lines, which are the target objects with obvious contrast with the black screen. It can be seen that in each task block 201, the preset screen sight constraint area always exists to constrain the subject's gaze range.

[0060] Taking a white vertical solid line as the target object is only an example and does not constitute a limitation on the target object of the preset screen sight constraint area. For example, the target object can also be a white dotted line, a red solid line, or a blue solid line, and there is no specific limitation on this.

[0061] like Figure 2 As shown, the white vertical solid line near the left side of the screen is at a 15% position relative to the left side of the screen, and the white vertical solid line near the right side of the screen is at a 85% position relative to the left side of the screen. This is only used as an example. The preset screen sight line constraint area can be defined based on the 10%-40% position between the two opposite sides of the screen and the 60%-90% position between the two opposite sides of the screen. This can not only constrain the subject's sight line within the preset screen sight line constraint area, but also avoid the sight line being concentrated on the center of the screen so that effective eye movement information cannot be collected, which is beneficial to controlling the subject's sight gaze position, reducing artifacts brought to EEG data by free eye movements, and ensuring that the data is easy to analyze and compare.

[0062] For example, Figure 2 As shown, the first visual element 204 is displayed at the center of the preset screen sight constraint area; the first mark 205 and the second mark 207 are displayed at the center of the preset screen sight constraint area; the second visual element 206 is displayed on the target object on one side of the preset screen sight constraint area, preferably at the center of the target object. In this way, when the subject performs the task, the sight is constrained to the preset screen sight constraint area, and the second visual element is presented on the target object during the third time period, which is conducive to controlling the gaze position of the subject's positive and negative saccades, and is conducive to improving the accuracy of the collected eye movement information, reducing the artifacts of free eye movement brought to the EEG data, and thus facilitating the collection of effective and accurate eye movement information and EEG data.

[0063] In some other embodiments of the present application, positive saccade trials and anti-saccade trials are pseudo-randomly performed alternately in a preset screen sight constraint area, and the subject's eye movement information about the location of the second visual element within a preset third time period in each trial is collected. When the subject starts to perform tasks in each task block, the preset screen sight constraint area will be presented on the screen, so as to constrain the subject's sight within the preset screen sight constraint area. For example, Figure 2As described above, when the second visual element 206, i.e., a gray dot, is presented on the target object on one side of the preset screen line of sight constraint area during the preset third time period, the subject performs an eye movement action of fixating on the position of the gray dot. At this time, an eye tracking device such as an eye tracker can be used to collect the subject's eye movement information during this period.

[0064] After the eye movement information is collected, the processor is used to obtain the eye movement response result of the subject to the second visual element according to the identification in the second time period based on the collected eye movement information, thereby obtaining the behavioral data of the subject when performing the task.

[0065] like Figure 2 As shown, after the gray dots in the third time period of the positive saccade trial 202 disappear, the white dots in the first time period of the anti-saccade trial 203 begin to appear in the preset screen sight constraint area on the screen. That is, in the process of executing a task block using the display, at the end of the third time period, the first visual element presentation of the first time period is jumped to, so that the subject can complete each trial in sequence according to the visual elements and logos presented in the preset screen sight constraint area on the screen.

[0066] Alternatively, if the subject is performing the last third time period task in the last trial of a task block 201, then after the subject completes this task, that is, after the gray dots and target objects in the preset screen sight constraint area on the screen disappear, a cross pattern of the first time interval appears on the screen, and the subject is resting for the first time interval. In other words, after a task block is completed using the display, at the end of the third time period, the display jumps to the third visual element presentation of the first time interval. The third visual element has different appearance features from the first and second visual elements, so that the subject can easily know that a rest is about to be taken based on the third visual element. Figure 2 The third visual element is a cross pattern, the first visual element is a white dot, and the second visual element is a gray dot. This is only used as an example and does not constitute a limitation on the specific solution.

[0067] Returning to the embodiment of the present application, in step S102, the subject's eye movement information within a preset third time period in each trial, the fNIRS detection data of the brain during the first time interval and in each task block, and the EEG data within a preset third time period from before the second time period in each trial to after the second time period which is shorter than the first time period are collected.

[0068] Here, the step of collecting eye movement information, brain fNIRS detection data, and EEG data in S102 can be executed simultaneously with the task block of S101, that is, while using the display to provide the subject with the task block, the above three-modal data of the subject can be collected, thereby realizing data collection under the three-modal paradigm.

[0069] Specifically, the brain fNIRS detection data during the first time interval and in each task block represents the brain fNIRS detection data of each task block and during the first time interval adjacent to the task block. By collecting the brain fNIRS detection data of each first time interval, the collected brain fNIRS detection data can be used as a baseline in the analysis process of the brain fNIRS detection data of its adjacent time period, so as to accurately analyze the changes in the brain fNIRS detection data of the subject in each task block.

[0070] Specifically, the EEG data is collected from the time before the second time period in each trial that is less than the length of the first time period to the preset third time period after the second time period, that is, the collection of EEG data is no earlier than the first time period and no later than the third time period. In this way, the EEG data of the predetermined time after the start of each time period in each trial can be used as a baseline, so as to accurately analyze the changes in the EEG data of the subject when responding to each trial. The predetermined time can be set according to actual experience, which can be 100ms, 200ms, and is not specifically limited here.

[0071] In some embodiments, eye movement information of the subject is collected when performing the task in each trial, and the eye movement information at least includes accuracy characterization parameters and eye saccade reaction time.

[0072] The accuracy characterization parameter may be, for example, the accuracy of the saccade direction, and the accuracy characterization parameter may reflect whether the subject can accurately focus or suppress the gaze point on the target stimulus.

[0073] The saccadic reaction time is related to the speed at which the subject processes visual information. If the saccadic reaction time is short, the subject can concentrate quickly. If the saccadic reaction time is long, the subject may have problems concentrating.

[0074] Based on the collaborative interaction method for collecting eye movement information, brain fNIRS detection data and EEG data provided in the embodiment of the present application, the display is used for the subject to perform a preset number of task blocks in sequence at a preset first time interval, and pseudo-randomly alternately perform positive eye saccade trials and anti-eye saccade trials in each task block, so that high-quality effective information can be collected for the brain fNIRS detection data, EEG data and eye movement information. Specifically, for brain fNIRS detection data, it is sufficient to collect effective and differentiated brain fNIRS detection data; for EEG data, it effectively reduces the subject's prediction and practice effects on stimulation, so that each trial stimulation can cause the maximum EEG response; for eye movement information, it can meet the use requirements of brain function evaluation in different dimensions while meeting the signal collection requirements of EEG equipment and the eye movement information collection requirements, so that the subject can simultaneously perform high-quality collection of eye movement-fNIRS-EEG trimodal data when executing the above task blocks.

[0075] In various embodiments of the present application, the subjects may include patients with mental illness or subjects with a tendency to mental illness. The collaborative interaction method for collecting eye movement information, brain fNIRS detection data and EEG data provided in the embodiments of the present application is particularly suitable for evaluating the brain function status of patients with mental illness or subjects with a tendency to mental illness, and for performing auxiliary brain function evaluation on patients with mental illness or subjects with a tendency to mental illness to obtain brain function evaluation results.

[0076] The brain function assessment result is an objective biological indicator of the subject, and the brain function assessment result can be combined with other auxiliary indicators to assess the mental state / brain function of the subject. For example, the brain function assessment result can be expressed as a predicted value representing the normal degree of brain function, a level of concern indicating whether a doctor needs to pay special attention, etc., without limitation.

[0077] Specifically, a rule-based data analysis method or a model-based data analysis method can be used to obtain the brain function assessment result of the subject based on the collected eye movement information, EEG data and brain fNIRS detection data of the subject using the processor: The rule-based data analysis method specifically includes: performing brain function assessment according to pre-established usage rules of eye movement information, EEG data, and brain fNIRS detection data.

[0078] The usage rules include the order of using data under different modalities, the threshold value representing the normal brain function of the subject, etc., which are not limited and can be set by the user.

[0079] Exemplarily, a first screening can be performed on the subject based on the EEG data and the brain fNIRS detection data. If the first screening result of the subject shows that the subject does not meet the preset normal standard of the EEG data and / or the normal standard of the brain fNIRS detection data, a second screening can be performed on the subject based on the eye movement information.

[0080] It is also possible to first perform a first screening on the subject based on the brain fNIRS detection data, and if the first screening result of the subject shows that the subject does not meet the preset normal standard of the brain fNIRS detection data, perform a second screening on the subject based on the EEG data, and if the first screening result and the second screening result show that the subject does not meet the preset normal standard of the EEG data and the normal standard of the brain fNIRS detection data, perform a third screening on the subject based on the eye movement information. In this way, the brain function assessment result is obtained according to each screening result.

[0081] Specifically, the specific method for obtaining the brain function assessment results based on the results of each screening includes: setting the brain function assessment results of the subjects who do not meet the preset normal standards for eye movement information as key attention; setting the brain function assessment results of the subjects who meet the preset normal standards for eye movement information but do not meet the preset normal standards for EEG data and / or normal standards for brain fNIRS detection data as general attention; setting the brain function assessment results of the subjects who meet the preset normal standards for eye movement information and the preset normal standards for EEG data and normal standards for brain fNIRS detection data as no need for attention.

[0082] In this way, by presetting the rules for using data under different modalities, the brain function status of the subject can be preliminarily divided, so that the subsequent assessment of the subject's mental state can be more targeted.

[0083] In some embodiments, the model-based data analysis method specifically includes: synchronously analyzing the data of the subject under three modalities based on a pre-trained model to obtain brain function assessment results.

[0084] Specifically, the labels used in the process of training the model may include at least one of the evaluation results such as the subjective evaluation results of the doctor and the evaluation results of the scale under examination. Based on the preset splicing order, the feature values ​​corresponding to the three modalities are spliced ​​to obtain the features to be analyzed, so as to classify and process the features to be analyzed using the pre-trained model to obtain the brain function evaluation results of the subject. Among them, the three modalities are eye movement modality, fNIRS modality and EEG modality.

[0085] In some embodiments, the characteristic value corresponding to the eye movement modality can be obtained by extracting features from indicators such as accuracy characterization parameters, saccade reaction time, and intra-individual variability of saccade reaction time in the eye movement information; the characteristic value corresponding to the fNIRS modality can be obtained by extracting features from the change in the brain fNIRS detection data in each task block compared with the brain fNIRS detection data during the first time interval; the characteristic value corresponding to the EEG modality can be obtained by extracting features from the EEG data in each task block compared with the change in the EEG data at a predetermined time after the start of each time period in each trial, and can also be obtained by performing time domain analysis on the EEG data in each task block.

[0086] In this way, the subjects only need to perform one trimodal task paradigm to achieve accurate collection of eye movement information, brain fNIRS detection data, and EEG data, so that the brain function of the subjects, especially those with mental illness or those with a tendency to mental illness, can be accurately evaluated. It also makes it easier for doctors to distinguish patients with mental disorders based on the objective biological indicators obtained from the evaluation, and to establish a more accurate classification model for the subsequent diagnosis of cognitive dysfunction.

[0087] In some other embodiments of the present application, the processor can be used to obtain the brain function auxiliary assessment results of the subject in schizophrenia, bipolar disorder, major depressive disorder and ADHD based on the collected eye movement information, EEG data and brain fNIRS detection data. For example, ADHD patients have difficulties in controlling the generation of saccades, and are more likely to have direction errors in anti-saccade trials. Moreover, the activation generated by ADHD patients in the dorsolateral prefrontal cortex is larger than that of the healthy group. Because ADHD patients themselves have defects in executive functions such as attention and inhibition control, when performing anti-saccade tasks, they need to make more efforts to suppress natural saccade reactions and make correct anti-saccade actions. And the enhanced activation of the dorsolateral prefrontal cortex may be a compensatory method adopted by the brain to make up for the deficiency of this executive function.

[0088] Schizophrenia (SZ) patients have a larger P100 amplitude induced by antisaccade conditions in the left temporal-occipital region. Because SZ patients may have defects in inhibitory control, the brain needs to work harder, that is, activate more neurons (manifested by a larger P100 amplitude) to overcome this difficulty and successfully perform the antisaccade task. Major depressive disorder (MDD) patients showed a higher antisaccade error rate than the healthy group, and the expected activation of the dorsolateral prefrontal cortex and inferior parietal cortex was weakened. The possible reason is that there is a problem with the brain neuroregulatory mechanism of MDD patients, which makes it difficult for them to effectively inhibit this reflexive saccade. This decline in inhibition may reflect their defects in cognitive control. Bipolar disorder (BD) patients showed a higher antisaccade error rate and longer reaction time than the healthy group, which indicates that BD patients have defects in the cognitive control ability to inhibit automatic reactions.

[0089] In this way, the eye movement information combined with the brain fNIRS detection data and EEG data can help doctors understand the brain function status of the subject from multiple dimensions, and can provide more comprehensive brain function activity information, which is conducive to the assessment of the brain function status of the subject from multiple dimensions, and then accurately assess the brain function of the subject. At the same time, the acquired eye movement information, brain fNIRS detection data, and EEG data can objectively and comprehensively reflect the impact of the different brain function conditions of the subject on each modality data, and can be used as effective data for combined auxiliary analysis of the subject's brain function status. In some embodiments of the present application, a terminal for providing a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data is provided, such as Figure 3 As shown, the terminal 300 is configured to use the processor 301 to sequentially execute a preset number of task blocks for the subject at a preset first time interval, and pseudo-randomly perform positive eye saccade trials and anti-eye saccade trials in each task block. The execution of positive eye saccade trials includes sequentially presenting a first visual element in a preset first time period, presenting a first mark indicating positive eye saccade in a preset second time period, and presenting a second visual element in a preset third time period; the execution of anti-eye saccade trials includes sequentially presenting a first visual element in a preset first time period, presenting a second mark indicating anti-eye saccade in a preset second time period, and presenting a second visual element in a preset third time period. The terminal 300 may also include a display 302 to display each visual element and mark in sequence on the screen of the display 302 in a preset order. The subject follows the task prompts presented on the display 302 to perform the tasks of the preset number of task blocks.

[0090] Among them, the fNIRS detection data of the brain during the first time interval and in each task block, and the EEG data in a preset third time period from before the second time period in each trial to after the second time period which is shorter than the first time period, are used for collaborative analysis with the eye movement information of the subject in the preset third time period in each trial to obtain the auxiliary assessment results of the subject's brain function.

[0091] Figure 4 A schematic diagram of a data acquisition system according to an embodiment of the present application is shown. The data acquisition system 400 includes an eye movement information acquisition device 401, an fNIRS device 402, and an EEG device 403, each of which is used to execute the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data as described in various embodiments of the present application.

[0092] The processor may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0093] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0094] The exemplary methods described in this application can be implemented at least in part by a machine or computer. In some embodiments, a computer-readable storage medium stores a computer program, which, when executed, implements the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data as described in various embodiments of this application.

[0095] In some other embodiments, a computer-readable storage medium stores a computer program, and when the computer program is executed, a processor is used to provide a trimodal paradigm of eye movement information, brain fNIRS detection data, and EEG data, specifically including sequentially executing a preset number of task blocks for the subject at a preset first time interval, pseudo-randomly alternatingly executing positive saccade trials and anti-saccade trials in each task block, executing positive saccade trials including sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; executing anti-saccade trials including sequentially presenting a first visual element within a preset first time period, presenting a second mark indicating an anti-saccade within a preset second time period, and presenting a second visual element within a preset third time period.

[0096] The implementation of such a method may include software code, such as microcode, assembly language code, high-level language code, etc. Various software programming techniques may be used to create various programs or program modules. For example, a program portion or program module may be designed with or by means of Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules may be integrated into a computer system and / or a computer-readable medium. Such software code may include computer-readable instructions for executing various methods. The software code may form a part of a computer program product or a computer program module. In addition, in an example, the software code may be tangibly stored on one or more volatile, non-temporary or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (such as optical disks and digital video disks), cassettes, memory cards or memory sticks, random access memories (RAMs), read-only memories (ROMs), etc.

[0097] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims are to be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the practice of this application, and the examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0098] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a disclosed feature that is not required to be protected is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a particular disclosed embodiment. Thus, the claims are incorporated herein as examples or embodiments in the specific embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the attached claims and the full scope of equivalent forms granted by these claims.

[0099] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and EEG data, characterized in that: The collaborative interaction method comprises: Using a display, the subject is asked to sequentially perform a preset number of task blocks at a preset first time interval, and pseudo-randomly and alternately perform positive saccade trials and anti-saccade trials in each task block, wherein the positive saccade trial comprises sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; and the anti-saccade trial comprises sequentially presenting a first visual element within a preset first time period, presenting a second mark indicating an anti-saccade within a preset second time period, and presenting a second visual element within a preset third time period; The eye movement information of the subject within the preset third time period in each trial, the fNIRS detection data of the brain during the first time interval and in each task block, and the EEG data within the preset third time period from before the second time period in each trial to after the second time period which is shorter than the first time period are collected.

2. The collaborative interaction method according to claim 1, characterized in that: For a complete task, 2-8 task blocks are executed in sequence at a first time interval of 10s-60s, and the duration of each task block is 10s-60s.

3. The collaborative interaction method according to claim 2, characterized in that: In each task block, the positive saccade trials and the anti-saccade trials were pseudo-randomly alternating with a ratio of 50±20% for the positive saccade trials and a ratio of 50±20% for the anti-saccade trials.

4. The collaborative interaction method according to claim 3, characterized in that: In the positive saccade trial, presenting the first mark to instruct the subject to perform an action of fixating the position of the second visual element within a preset third time period; In the antisaccade trial, presenting the second mark to instruct the subject to perform an action of fixating a position equidistant from a position where the second visual element is located within a preset third time period; Wherein, the positive saccade trial includes a positive saccade to the left reaction condition and a positive saccade to the right reaction condition, and / or includes a positive saccade to the upper reaction condition and a positive saccade to the lower reaction condition; The antisaccade trial includes an antisaccade look to the left reaction condition and an antisaccade look to the right reaction condition, and / or includes an antisaccade look to the top reaction condition and an antisaccade look to the bottom reaction condition.

5. The collaborative interaction method according to claim 4, characterized in that: In each task block, pseudo-randomly and alternately perform the positive saccade look-to-left reaction condition, the positive saccade look-to-right reaction condition, the anti-saccade look-to-left reaction condition, and the anti-saccade look-to-right reaction condition, wherein the positive saccade look-to-left reaction condition, the positive saccade look-to-right reaction condition, the anti-saccade look-to-left reaction condition, and the anti-saccade look-to-right reaction condition are each performed at least once; And / or, in each task block, the positive saccade looking up reaction condition, the positive saccade looking down reaction condition, the anti-saccade looking up reaction condition and the anti-saccade looking down reaction condition are pseudo-randomly and alternately executed, wherein the positive saccade looking up reaction condition, the positive saccade looking down reaction condition, the anti-saccade looking up reaction condition and the anti-saccade looking down reaction condition are each executed at least once.

6. The collaborative interaction method according to any one of claims 1 to 5, characterized in that: In each trial, the first time period is 200ms-2000ms, the second time period is 500ms-2000ms, and the third time period is 1000ms-2500ms.

7. The collaborative interaction method according to claim 6, characterized in that: When executing a positive saccade trial in the task block, the first visual element, the first identifier, and the second visual element of the positive saccade trial are presented in a preset screen sight constraint area; when executing an antisaccade trial in the task block, the first visual element, the second identifier, and the second visual element of the antisaccade trial are presented in a preset screen sight constraint area; wherein the preset screen sight constraint area is defined based on a 10%-40% position between two opposite sides of the screen and a 60%-90% position between two opposite sides of the screen, and the two side boundaries of the preset screen sight constraint area are respectively presented as target objects with obvious contrast with the screen color, and the second visual element is presented on one of the target objects.

8. The collaborative interaction method according to claim 6, characterized in that: The first visual element and the second visual element have different appearance features, and the first logo and the second logo have different appearance features.

9. The collaborative interaction method according to claim 7, characterized in that: Pseudo-randomly alternating positive saccade trials and anti-saccade trials in the preset screen sight constraint area, collecting the subject's eye movement information for the location of the second visual element within a preset third time period in each trial, and using a processor to obtain the subject's eye movement response result to the second visual element according to the identifier within the second time period based on the eye movement information; In the process of executing a task block using the display, jumping to the first visual element presentation of the first time period at the end of the third time period; Alternatively, after a task block is executed using the display, the display jumps to the third visual element presentation of the first time interval at the end of the third time period, wherein the third visual element has different appearance features from the first visual element and the second visual element.

10. The collaborative interaction method according to any one of claims 1 to 5, characterized in that: The eye movement information at least includes accuracy characterization parameters and eye movement reaction time.

11. The collaborative interaction method according to any one of claims 1 to 5, characterized in that: Before executing the first task block, a second time interval is executed, and the duration of the second time interval is 10s-60s.

12. A terminal for providing a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data, characterized in that: The terminal is configured to utilize a processor to: The subject is asked to perform a preset number of task blocks in sequence at a preset first time interval, and pseudo-randomly and alternately perform positive saccade trials and anti-saccade trials in each task block, wherein the positive saccade trials include sequentially presenting a first visual element in a preset first time period, presenting a first mark indicating positive saccades in a preset second time period, and presenting a second visual element in a preset third time period; Executing the antisaccade trial includes sequentially presenting a first visual element in a preset first time period, presenting a second mark indicating an antisaccade in a preset second time period, and presenting a second visual element in a preset third time period; Among them, the fNIRS detection data of the brain during the first time interval and in each task block, and the EEG data in a preset third time period from before the second time period in each trial to after the second time period which is shorter than the first time period, are used for collaborative analysis with the eye movement information of the subject in the preset third time period in each trial to obtain the auxiliary assessment results of the subject's brain function.

13. A data acquisition system, characterized in that: The data acquisition system includes an eye movement information acquisition device, an fNIRS device, and an electroencephalogram device, each of which is used to execute the collaborative interaction method for collecting eye movement information, brain fNIRS detection data and electroencephalogram data as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, it implements the collaborative interaction method for collecting eye movement information, brain fNIRS detection data, and electroencephalogram data as described in any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the processor provides a three-modal paradigm of eye movement information, brain fNIRS detection data, and EEG data, specifically including: The subject is required to sequentially perform a preset number of task blocks at a preset first time interval, and pseudo-randomly alternately perform positive saccade trials and anti-saccade trials in each task block, wherein the execution of the positive saccade trial includes sequentially presenting a first visual element within a preset first time period, presenting a first mark indicating a positive saccade within a preset second time period, and presenting a second visual element within a preset third time period; and the execution of the anti-saccade trial includes sequentially presenting the first visual element within the preset first time period, presenting a second mark indicating an anti-saccade within the preset second time period, and presenting the second visual element within the preset third time period.

Images