Cooperative interaction method for collecting near-infrared detection data and electroencephalogram data, terminal, data collection system and storage medium

Abstract

The invention provides a collaborative interaction method for acquiring near-infrared detection data and electroencephalogram data, a terminal, a data acquisition system and a storage medium. The collaborative interaction method comprises the following steps: executing a first preset number of task blocks in sequence at a preset first time interval, and executing a second preset number of trials in each task block in sequence; the data collected in the collaborative interaction process comprises near-infrared detection data of the subject in each task block and during an adjacent first time interval, and electroencephalogram data in a preset third time period before and after presentation of each number in each test number. Thus, the brain function condition of the subject can be objectively evaluated through one interaction process, the sequence of the single number used for judgment in the interaction process is the preset pseudo-random sequence, the interaction difficulty is more easily regulated and controlled, the difficulty of different times of interaction of the same subject can be unified conveniently, and the interaction efficiency is improved. And the accuracy of a brain function evaluation result is ensured.

Description

Technical Field

[0001] The present application relates to the technical field of brain function assessment, and in particular to a collaborative interaction method, terminal, data acquisition system and storage medium for collecting near-infrared detection data and electroencephalogram data. Background Art

[0002] Schizophrenia (SZ) patients have obvious working memory impairment, and unipolar disorder (UD) and bipolar disorder (BD) patients also have working memory impairment. Although the current fNIRS research results show that there are differences in blood oxygen response between UD and BD patients when performing working memory tasks, it is impossible to rely on a single brain function assessment method to accurately assess the working memory status of the subjects.

[0003] At present, researchers have developed a variety of brain function assessment methods, trying to use different brain function assessment equipment to evaluate the brain function status of the subjects. However, due to the different specific needs of different brain function assessment equipment during use, for example, the blood oxygen signal collected based on fNIRS is a slow signal, and the EEG signal collected based on EEG is a fast signal. Therefore, only a single brain function assessment method is often used. In order to obtain evaluation indicators under multiple dimensions, data under multiple modalities are usually collected separately in separate processes. In addition, subjects with a tendency to disease and patients in the stage of evaluating treatment effects often need multiple assessments. The above method not only has low work efficiency, but also the difficulty of the task paradigm in different assessments is difficult to maintain uniformity. Therefore, it is impossible to obtain an objective and comprehensive assessment of the brain function status of the subject. Summary of the invention

[0004] This application is proposed in view of the above-mentioned technical problems existing in the prior art. This application aims to provide a collaborative interaction method, terminal, data acquisition system and storage medium for collecting near-infrared detection data and EEG data, which can provide a dual-modal task paradigm of near-infrared detection data and EEG data, and can realize intuitive, convenient and efficient collaborative interaction with the subject using a single display. In the process of collaborative interaction, near-infrared detection data and EEG data of different time periods are obtained, and the acquired near-infrared detection data and EEG data can be coordinated to objectively and comprehensively reflect the influence of different brain function conditions of the subject on each modality data.

[0005] According to a first scheme of the present application, a collaborative interaction method for collecting near-infrared detection data and electroencephalogram (EEG) data is provided, the collaborative interaction method comprising: using a display to sequentially execute a first preset number of task blocks for a subject at a preset first time interval, and sequentially executing a second preset number of trials in each task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period; receiving a judgment result of the subject in the second time period as to whether the number of the current trial is the same as the number of the previous nth trial, wherein n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900ms, so that each candidate time period maintains a predetermined proportion, and the sequence of single numbers appearing in the first preset number of task blocks is a preset pseudo-random sequence; the data collected during the collaborative interaction process include the near-infrared detection data of the subject during each task block and the adjacent first time interval, and the electroencephalogram (EEG) data in a preset third time period from before to after the presentation of each number in each trial.

[0006] According to a second scheme of the present application, a terminal for providing a dual-modal paradigm based on fNIRS and EEG is provided, wherein the terminal is configured to use a processor to: sequentially execute a first preset number of task blocks for a subject at a preset first time interval, and sequentially execute a second preset number of trials in each task block, wherein each trial sequentially includes pseudo-randomly presenting a single number in a first time period, presenting a fixation point in a variable second time period, and receiving a judgment result of the subject as to whether the number of the current trial is the same as the number of the previous nth trial, where n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900 ms, so that each candidate time period maintains a predetermined proportion; wherein the near-infrared detection data of the subject during each task block and the adjacent first time interval is used for collaborative analysis with the electroencephalogram data in a preset third time period from before to after the presentation of each number in each trial to obtain an auxiliary assessment result of the subject's brain function.

[0007] According to the third scheme of the present application, a data acquisition system based on a dual-modal paradigm of fNIRS and EEG is provided, wherein the data acquisition system comprises at least an fNIRS device and an EEG device, and each device is used to perform data acquisition when executing the collaborative interaction method for acquiring near-infrared detection data and EEG data as described in the various embodiments of the present application.

[0008] According to the fourth scheme of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the collaborative interaction method for collecting near-infrared detection data and electroencephalogram data as described in the various embodiments of the present application is implemented.

[0009] According to the fifth scheme of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed, a dual-modal paradigm of near-infrared detection data and electroencephalogram data is provided via a processor, specifically comprising: executing a first preset number of task blocks in sequence at a preset first time interval for a subject, and executing a second preset number of trials in sequence in each task block, wherein each trial sequentially comprises presenting a single number pseudo-randomly in a first time period, presenting a fixation point in a variable second time period, and receiving a judgment result of the subject as to whether the number of the current trial is the same as the number of the previous nth trial, wherein n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900 ms, so that each candidate time period maintains a predetermined proportion.

[0010] Compared with the prior art, the beneficial effects of the embodiments of the present application are:

[0011] The collaborative interaction method for collecting near-infrared detection data and EEG data provided by the embodiment of the present application uses a display to perform a first preset number of task blocks in sequence for the subject at a preset first time interval, and sequentially performs a second preset number of trials in each task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period. In this way, for near-infrared detection data, the sequence of single numbers appearing in the first preset number of task blocks is a preset pseudo-random sequence, which is sufficient to collect effective and differentiated near-infrared detection data; for EEG data, a single number is pseudo-randomly presented in the first time period, and the second time period is selected in a pseudo-random manner from a group of candidate time periods of more than 900ms, which can reduce the expected effect of the subject on each number, and is conducive to collecting high-quality EEG data. Therefore, when the subject executes the above-mentioned task blocks, the high-quality collection of near-infrared detection data-EEG data dual-modal data can be simultaneously performed, and the data collected in each mode can be used for brain function status evaluation, especially for evaluating the working memory state of the subject.

[0012] The collaborative interaction method provided by the embodiment of the present application enables the examinee to perform only one task paradigm on a single terminal to achieve accurate collection of near-infrared detection data and electroencephalogram data, so that the examinee, especially the examinee with mental illness or the examinee with mental illness tendency, can be accurately evaluated for brain function, and it is also convenient for doctors to distinguish patients with mental disorders according to the objective biological indicators obtained by the evaluation. On the other hand, since only one task paradigm is required to complete the accurate collection of two types of data at the same time, and the sequence of single numbers used for judgment in the task paradigm is a preset pseudo-random sequence, the difficulty of the task paradigm is controllable. Compared with executing two single-modal data acquisition paradigms separately, executing only one task paradigm can not only improve the data collection efficiency, but also make it easier to regulate the difficulty of the task paradigm, and facilitate the unification of the difficulty of the same examinee in different task paradigms, thereby ensuring the accuracy and consistency of the brain function evaluation of the examinee, especially for the evaluation of the treatment effect of patients with related diseases, the task paradigm with consistent difficulty can accurately evaluate the treatment effect of patients with related diseases, reduce the interference caused by the different difficulties of the task paradigm, and improve the accuracy of the evaluation results.

[0013] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. Similar reference numerals with letter suffixes or different letter suffixes may represent different examples of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not by way of limitation, and together with the description and claims, are used to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary and are not intended to be exhaustive or exclusive embodiments of the present method, apparatus, system, or non-transitory computer-readable medium having instructions for implementing the method.

[0015] Figure 1 A flow chart of a collaborative interaction method for collecting near-infrared detection data and electroencephalogram data according to an embodiment of the present application is shown.

[0016] Figure 2 A schematic diagram showing task prompts presented on a screen on a display during a formal experiment in a collaborative interaction method for collecting near-infrared detection data and electroencephalogram data provided in an embodiment of the present application.

[0017] Figure 3A schematic diagram of a terminal for providing a dual-modality paradigm based on fNIRS and EEG according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods, but are not intended to limit the present application.

[0019] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.

[0020] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined here. Techniques and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the techniques and equipment should be considered as part of the specification.

[0021] Figure 1 A flowchart of a collaborative interaction method for collecting near-infrared detection data and EEG data according to an embodiment of the present application is shown. Specifically, the method shown in step S101-step S103 is executed. In this embodiment, the arrows shown in the figure for each step are only examples of the execution order, not limitations. The technical solution of the present application is not limited to the execution order described in the embodiment. The various steps in the execution order can be combined, decomposed, or changed in order, as long as the logical relationship of the execution content is not affected.

[0022] Exemplarily, before the subject performs the near-infrared detection data-EEG data dual-modal paradigm, the subject can be provided with a head cap suitable for collecting near-infrared detection data and EEG data. The installation position of the near-infrared signal acquisition probe and the installation position of the EEG electrodes configured in the head cap can be set by the user based on prior medical knowledge, and no specific limitation is made to this.

[0023] After the subject wears the head cap suitable for collecting near-infrared detection data and EEG data, the near-infrared detection data-EEG data dual-modal task can be started to simultaneously collect the near-infrared detection data and EEG data of the subject when performing the dual-modal task.

[0024] Specifically, in step S101, a display is used to sequentially perform a first preset number of task blocks for the subject at a preset first time interval, and a second preset number of trials are sequentially performed in each task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period.

[0025] In step S102, a judgment result of the subject as to whether the number of the current trial is the same as the number of the previous n-th trial is received within the second time period, wherein n is 1 or 2, and each second time period in each task block is selected in a pseudo-random manner from a group of candidate time periods of more than 900 ms, so that each candidate time period maintains a predetermined proportion, and the sequence of single numbers appearing in the first preset number of task blocks is a preset pseudo-random sequence.

[0026] The display can be understood as a device with a display function, which can be a device with both data processing and display functions, such as a laptop computer, a tablet computer, a smart phone, etc., or it can be an external display screen.

[0027] Specifically, task prompts requiring the subject to perform a dual-modal task can be displayed on the display, so that the subject can perform the corresponding dual-modal task according to the task prompts displayed on the display, and the near-infrared data acquisition module and the EEG data acquisition module can be used to simultaneously collect near-infrared detection data and EEG data of the subject when performing the dual-modal task.

[0028] Figure 2 The schematic diagram of the task prompts presented on the screen of the display during the formal experiment is shown. Before presenting the first task block 201, a cross symbol is presented on the screen to remind the subject to take a rest before performing the first task block 201. The rest time can be 20s, 30s, 40s, or 50s, which is not limited to this, so that the subject knows that the task is about to start and is mentally prepared to perform the task.

[0029] like Figure 2 After the subject completes the first task block 201, the first time interval is performed. In some other embodiments, the first time interval is 10s-60s, which is conducive to allowing the subject to take a rest during the first time interval to adjust the state and concentrate on performing subsequent tasks.

[0030] In some other embodiments of the present application, the first preset number is 1-9, and the second preset number is 15-50. In this way, a better blood oxygen signal response can be caused, and the requirements for extraction and analysis of near-infrared detection data can be met to obtain accurate and effective near-infrared detection data.

[0031] in, Figure 2 Only the first task block 201 and the first time interval adjacent to the first task block 201 are shown, and only two trials in the first task block 201 are shown, and the remaining task blocks and the first time intervals adjacent to the task blocks, as well as the other trials in each task block 201 are indicated by ellipsis, and, Figure 2 The content shown is only for facilitating the understanding of the technical solution and does not constitute a limitation on the specific solution.

[0032] like Figure 2 As shown, each task prompt is presented in sequence on the screen of the display. For the first trial 202 in the first task block 201, a single number 2 is presented in the first time period, and a cross symbol is presented in the second time period. For the second trial 203 in the first task block 201, a single number 8 is presented in the first time period, and a cross symbol is presented in the second time period as a fixation point. For the third trial in the first task block 201 (i.e., the next trial of the second trial 203, not shown), a single number 8 is presented in the first time period, and the content presented in the second time period and the remaining trials are all represented by ellipsis. Among them, the fixation point can also be other symbols such as dots and boxes, which are not limited to this.

[0033] In some other embodiments of the present application, the first time period is 217-1017ms. For example, the first time period may be 250ms, 500ms, 517ms, 600ms, 657ms, or 1000ms. There is no limitation on this and the user may set the first time period. For example, Figure 2 The first time period in each task block 201 shown may be 517 ms, ie, when the subject performs the task in each task block 201 , a single number is presented for 517 ms in each trial.

[0034] In this way, the subject can have sufficient time to see the number in the current trial within the first time period of at least 217ms, and the time of the first time period does not exceed 1017ms, avoiding the overall task taking too long due to a single trial taking too long, effectively saving the subject's time cost.

[0035] In some other embodiments of the present application, each candidate time period is selected from the range of 900-2100ms, and the deviation between each candidate time period is 50-600ms. Since the deviation between each candidate time period is 50-600ms, each second time period in each task block is selected from each candidate time period in a pseudo-random manner, which is conducive to avoiding interference from external regular noise, and is conducive to reducing the expected effect and obtaining high-quality EEG data.

[0036] In some other embodiments of the present application, the group of candidate time periods longer than 900ms includes a first candidate time period, a second candidate time period and a third candidate time period, the first candidate time period is 957-1357ms, the second candidate time period is 1357-1657ms, and the third candidate time period is 1657-2157ms.

[0037] Here, the subject has sufficient time, at least 957ms, to determine whether the number in the current trial is the same as the number in the previous n-th trial, and each candidate time period does not exceed 2157ms, thus avoiding the overall task taking too long due to a single trial taking too long, and effectively saving the subject's time cost.

[0038] For example, Figure 2 For each trial of each task block 201, the second time period can be selected in a pseudo-random manner from the three candidate time periods of 1657ms, 1457ms and 1257ms, and in each task block 201, each candidate time period maintains a predetermined ratio.

[0039] For each task block 201, the second time period of 1657ms has the same proportion, the second time period of 1457ms has the same proportion, and the second time period of 1257ms has the same proportion. For example, assuming that a complete assessment includes three task blocks 201, in these three task blocks 201, the ratio of the second time periods of 1657ms, 1457ms, and 1257ms is 1:1:1, and the number of trials included in these three task blocks 201 is also the same.

[0040] This helps to keep the same time length for each task block 201, making it easier to process and compare the near-infrared detection data.

[0041] Exemplarily, when n is 1, when the subject performs the second trial 203, he needs to determine whether the single number 8 appearing in the second trial 203 is the same as the single number 2 appearing in the first previous trial (ie, the first trial 202).

[0042] For example, when n is 2, when the subject performs the third trial, he needs to determine whether the single number 8 that appears in the third trial is the same as the single number 2 that appeared in the second trial before (ie, the first trial 202).

[0043] The examinee performs different trigger operations based on different judgment results. For example, if the judgment results are the same, the left button of the mouse can be pressed, and if the judgment results are different, the right button of the mouse can be pressed.

[0044] The trigger operation is not specifically limited, as long as the difference in the judgment result can be distinguished through different trigger operations. For example, the trigger operation can also be buttons with different indication directions or buttons with different colors displayed on the touch screen, which is only used as an exemplary description.

[0045] Specifically, in response to different trigger operations performed by the subject, the processor can be used to receive the subject's judgment result on whether the number of the current trial is the same as the number of the previous n-th trial within the second time period.

[0046] Specifically in this embodiment, for example, when n is 1, a single digit 8 is presented for 517ms in the second trial 203, and a cross symbol is presented for 1657ms on the next screen; a single digit 8 is presented for 517ms in the third trial, and a cross symbol is presented for 1257ms on the next screen. This is only used as an exemplary description and will not be listed one by one.

[0047] The fixation point is presented within a variable second time period, and the second time period is at least 900ms, which is sufficient to enable the subject to judge whether the number of the current trial is the same as the number of the nth trial before within the second time period, and to perform different trigger operations, so that the corresponding processing module responds to different trigger operations and receives the subject's judgment result on whether the number of the current trial is the same as the number of the nth trial before.

[0048] Furthermore, each second time period is selected in a pseudo-random manner from a group of candidate time periods of more than 900 ms, which can reduce the expected effect of the subject, induce a significant EEG response, avoid the influence of external regular noise, and improve the quality of the collected data.

[0049] In this embodiment, the sequence of single numbers appearing in the first preset number of task blocks is a preset pseudo-random sequence, and the pseudo-random sequence refers to a digital sequence generated according to a preset sequence generation rule. Each single number in each pseudo-random sequence is presented pseudo-randomly within a first time period according to a preset order. The pseudo-random sequence can be 2, 3, or 4, and there is no limitation on this. The single numbers in the pseudo-random sequence and the arrangement order of the single numbers can be set by the user or defined by the pseudo-random algorithm used by the user.

[0050] Among them, the original sequence can be generated according to the preset pseudo-random algorithm, and at least one sequence that meets the requirements can be found from the original sequence. For example, if there are 20 numbers in the pseudo-random sequence, then one sequence containing 20 numbers that meets the requirements can be found from the original sequence, or two sequences containing 10 numbers can be found, or four sequences containing 5 numbers can be found, and then spliced ​​to generate the final pseudo-random sequence. In the case of using multiple sequences to splice and generate a pseudo-random sequence, the spliced ​​pseudo-random sequence needs to meet the corresponding difficulty requirements, such as the proportion of the target sequence and the interference sequence contained therein needs to reach the predetermined proportion.

[0051] In addition, the generation model may be used to automatically generate a pseudo-random sequence based on conditions input manually. This is only used as an example, and there is no limitation on the source of the pseudo-random sequence.

[0052] Specifically, the digital sequence corresponding to each task block can be selected in a pseudo-random manner from multiple pseudo-random sequences. For example, when n is 1, the pseudo-random sequence can include the digital sequence 2,5,7,7,3,5,4,9,9,0,3,2,2,9,7,6,6,0,3,3, 6,6,5,3,2,3,1,1,7,8,8,4,5,5,3,2,2,1,0,9 and the digital sequence 3,7,7,8,2,3,6,6,5,2,1,1,6,4,4,8,0,0,6,7.

[0053] For example, when n is 2, the pseudo-random sequence may include the digital sequence 5,2,5,3,4,3,6,6,5,4,5,3,1,4,1,8,1,0,6,7, 3,7,2,3,2,1,7,1,0,1,4,7,1,7,7,3,0,3,5,5 and the digital sequence 0,5,5,0,4,3,4,7,5,1,5,2,0,2,1,8,1,7,6,7.

[0054] At this time, assuming that there are 5 task blocks in a complete task, and n is 1, then, two of the task blocks may correspond to the pseudo-random sequences of 2,5,7,7,3,5,4,9,9,0,3,2,2,9,7,6,6,0,3,3, two of the task blocks may correspond to the pseudo-random sequences of 6,6,5,3,2,3,1,1,7,8,8,4,5,5,3,2,2,1,0,9, and the last task block may correspond to the pseudo-random sequence of 3,7,7,8,2,3,6,6,5,2,1,1,6,4,4,8,0,0,6,7. This is only for illustrative purposes.

[0055] In some other embodiments, each pseudo-random sequence is presented in sequence corresponding to each task block in a fixed arrangement order. For example, there are three task blocks, and a first pseudo-random sequence, a second pseudo-random sequence and a third pseudo-random sequence are set respectively. The first pseudo-random sequence is executed for the first task block, the second pseudo-random sequence is executed for the second task block, and the third pseudo-random sequence is executed for the third task block.

[0056] This is only an example and does not constitute a limitation on the specific solution.

[0057] Based on the preset pseudo-random sequence, it can ensure that different subjects have controllable execution difficulty when performing the near-infrared detection data-EEG data dual-modal task paradigm, avoiding the situation where the task difficulty cannot be controlled due to the random presentation of digital sequences, which may cause the subject to be unable to fully and effectively perform the entire task paradigm. Moreover, based on the preset pseudo-random sequence, it is easier to regulate the difficulty of the task paradigm, which is convenient for unifying the difficulty of the same subject in different task paradigms, thereby ensuring the accuracy and consistency of the subject's brain function assessment. In addition, since subjects with mental illness or a tendency to mental illness have limited cooperation and patience when performing the task paradigm, by controlling the difficulty of the task paradigm, not only can subjects with mental illness or a tendency to mental illness be able to better complete the task paradigm, but also a more accurate and objective assessment of their brain function status can be achieved.

[0058] In some other embodiments of the present application, a target number or a non-target number is pseudo-randomly presented alternately as the single number during a first time period, wherein the target number represents a number that is the same as the number in the nth previous trial, and the non-target number represents a number that is different from the number in the nth previous trial, wherein the ratio of the target number in each task block is 10%-50%.

[0059] Specifically, take n as 1, and the pseudo-random sequence corresponding to the task block is the digital sequence 2,5,7,7,3,5,4,9,9,0,3,2,2,9,7,6,6,0,3,3 as an example for explanation. Among them, the target numbers are the second "7" in "7,7", the second "9" in "9,9", the second "2" in "2,2", the second "6" in "6,6", and the second "3" in "3,3". For example, for the second "7" in "7,7", the target number 7 is the same as the number "7" in the first trial. The non-target numbers are "5" in "2,5", "7" in "5,7", and "3" in "7,3", which are all different from the numbers in the first trial. The other non-target numbers are no longer listed one by one.

[0060] Among them, the task block with a target number ratio of 10%-50% is of moderate difficulty for the subjects, so that each stimulation of the target number can cause a significant EEG response, thereby effectively collecting EEG data and near-infrared detection data that truly reflect the differences in brain function status.

[0061] In some other embodiments of the present application, the same subject uses different pseudo-random sequences in different collaborative interactions, and the difficulty of each pseudo-random sequence is the same or similar. Specifically, when the same subject performs different task blocks, the pseudo-random sequences corresponding to each task block are different, which can prevent the subject from anticipating the target numbers that appear in each trial, thereby avoiding the expectation effect, and is conducive to collecting the subject's real near-infrared detection data and EEG data.

[0062] Among them, for example, by controlling the same or similar number of target numbers in each pseudo-random sequence, the difficulty of each pseudo-random sequence can be made the same or similar, so as to avoid the sudden appearance of a task block with higher difficulty or a task block with lower difficulty during the task execution of the examinee, so that the difficulty of each task block is maintained at a stable level acceptable to the examinee. If the difficulty of the task block fluctuates greatly, it is easy to cause emotional changes in the examinee, and even the examinee may have an emotional breakdown due to the sudden increase in the difficulty of the task and be unable to perform the complete task.

[0063] In some embodiments of the present application, each preset pseudo-random sequence includes a target sequence and an interference sequence, wherein the last digit of the target sequence is the same as the previous nth digit, the first digit and the last digit of the interference sequence are the same as the target digit, and the number of digits included in the interference sequence is different from the number of digits included in the target sequence.

[0064] Specifically, for example, when n is 1, the target sequence is BB (indicating that two identical numbers appear consecutively), and the interference sequences at this time include BAB (A and B are different numbers). For example, when n is 2, the target sequence is BAB (indicating that the last number is the same as the second number that appears before, and A and B are different numbers), and the interference sequences at this time include BB, BACB (B is a number different from A and C).

[0065] The interference sequence is similar to the target sequence structure, which can easily cause the subject to make a wrong judgment. For example, when n is 1, for the pseudo-random sequence 2, 5, 7, 7, 3, 5, 4, 9, 8, 9, 0, 3, 2, 2, 9, 7, 6, 5, 6, 0, 3, 3, the target sequence includes "7, 7", "2, 2", "3, 3", and the interference sequence includes "9, 8, 9", "6, 5, 6".

[0066] In this embodiment, the difficulty of each preset pseudo-random sequence is determined based on at least one of the proportion of the target sequence in the pseudo-random sequence, the proportion of the interference sequence, and the distribution of the target sequence in the pseudo-random sequence.

[0067] The proportion of the target sequence can be expressed as the ratio of the target sequence to the complete sequence of the same task block, and similarly, the proportion of the interference sequence can be expressed as the ratio of the interference sequence to the entire complete sequence of the same task block. The distribution of the target sequence in the pseudo-random sequence can be determined based on the intervals between the target sequences in the complete sequence.

[0068] In some other embodiments of the present application, each preset pseudo-random sequence also includes a continuous target sequence, wherein the continuous target sequence includes at least two adjacent target sequences. For example, when n is 2, for the pseudo-random sequence 0,5,5,0,4,3,4,7,2,3,2,3,0,2,1,8,1,7,6,7, the continuous target sequence is "2,3,2,3", and when the second "2" is presented, the subject will think that the target sequence has appeared. At this time, the number retained in the subject's brain is 2, resulting in a significant increase in the difficulty of judgment when the second "3" is presented, and it is easy to make a misjudgment, so the recognition difficulty when the continuous target sequence appears increases. In the case where n is 1, for the pseudo-random sequence 2,5,7,7,3,5,4,9,8,9,0,3,2,2,3,3,4,4,6,0,3,3, the continuous target sequence is "2,2,3,3,4,4", and in this case, the subject is likely to generate expectations based on the continuous target sequence. Therefore, for the case where n is 1, the continuous target sequence will reduce the recognition difficulty. That is, the effect of the continuous target sequence on the difficulty judgment of the pseudo-random sequence when n is 1 is opposite to the effect of the continuous target sequence on the difficulty judgment of the pseudo-random sequence when n is 2.

[0069] Therefore, the difficulty of each preset pseudo-random sequence is determined based on at least one of the proportion of the target sequence in the pseudo-random sequence, the proportion of the interference sequence, the distribution of the target sequence in the pseudo-random sequence, and the proportion of the continuous target sequence.

[0070] In some other embodiments of the present application, the proportion of target sequences in each preset pseudo-random sequence is 10%-50%, the proportion of interference sequences in the pseudo-random sequence does not exceed the proportion of target sequences, and the proportion of continuous target sequences does not exceed half of the proportion of interference sequences, so as to control the difficulty of the pseudo-random sequence within an acceptable range for the subjects, avoid excessive difficulty that causes the subjects to be unable to complete the entire task, and avoid excessively low difficulty that fails to induce accurate and effective EEG responses in the subjects.

[0071] By adjusting the proportion of target sequences, the proportion of interference sequences and the proportion of continuous target sequences, the difficulty of the pseudo-random sequence can be reasonably adjusted, so that the difficulty of each preset pseudo-random sequence is the same or similar, which is conducive to ensuring the consistency of evaluation of different subjects.

[0072] In some other embodiments of the present application, before the subject executes the task block, the training block is executed, and a third preset number of trials are performed for the subject using the display, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a second time period; receiving the subject's judgment result as to whether the number of the current trial is the same as the second previous number in the second time period, the third preset number being less than the second preset number; and determining the value of n based on the accuracy of the subject's judgment results in the training block, specifically including: when the accuracy of the subject's judgment results in the training block reaches a predetermined accuracy requirement, determining n to be 2, otherwise n is 1.

[0073] Preferably, when the subject performs the tasks of the training block, he or she performs at most one task of the training block, and the third preset number is less than the second preset number, so as to avoid the subject consuming too much energy in the process of performing the tasks of the training block, resulting in the inability to concentrate on performing subsequent task blocks.

[0074] The preset accuracy rate can be set by the user, for example, the preset accuracy rate can be 40% to 60%, such as 40%, 45%, 50%, 55%, 60%, etc. Taking the preset accuracy rate of 50% as an example, if the accuracy rate of the subject's judgment results in the training block is greater than or equal to 50%, it means that the subject can adapt to the difficulty of the task block with n=2, and if the accuracy rate is less than 50%, it means that the difficulty of the task block with n=2 is too difficult for the subject.

[0075] In this way, the value of n is determined based on the accuracy of the subject's judgment results in the training block, which is helpful for the subject to understand the task rules and can also provide task blocks with appropriate difficulty according to the brain function status of each subject.

[0076] In some other embodiments of the present application, the ratio of the single digits in the training block to the target digits is less than the ratio of the single digits in each task block to the target digits, the target digits represent the same digits as the nth digits before, and the non-target digits represent digits different from the nth digits before. In this way, the difficulty of the training block is less than that of the task block, which can not only enhance the self-confidence of the examinee in performing the task, but also help to induce a more obvious near-infrared signal response and EEG response by increasing the difficulty of the task block compared to the training block.

[0077] In some other embodiments of the present application, a first preset number of task blocks includes a first task block and a second task block, and a second preset number of trials are sequentially executed in each first task block, and each trial sequentially includes pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period; and receiving a judgment result of the subject in the second time period as to whether the number of the current trial is the same as the number of the previous first trial.

[0078] A second preset number of trials are sequentially executed in each second task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period; a judgment result of the subject as to whether the number of the current trial is the same as the number of the second previous trial is received in the second time period; wherein the execution priority of the first task block is greater than the execution priority of the second task block.

[0079] In the first task block, the subject is required to judge whether the number in the current trial is the same as the number in the first previous trial. In the second task block, the subject is required to judge whether the number in the current trial is the same as the number in the second previous trial. In other words, the difficulty of the first task block is less than that of the second task block.

[0080] Specifically, in some embodiments, the subject may not perform the tasks of the training block, but when performing the tasks of the task block, first perform the first task block of lower difficulty, and then perform the second task block of higher difficulty. For a complete assessment, including 2 task blocks, including 1 first task block and 1 second task block, the content prompt of the first task block is preferentially displayed on the display, so that the subject can preferentially perform the first task block, and then perform the second task block. In this way, the difficulty of the task blocks performed by the subject is gradually increased, which is easy for the subject to accept.

[0081] Back to the embodiment of the present application, in step S103, the data collected during the collaborative interaction process includes at least the near-infrared detection data of the subject during each task block and the adjacent first time interval, and the EEG data in a preset third time period from before to after each number is presented in each trial. Among them, the step of collecting near-infrared detection data and EEG data can be performed simultaneously with the task block in step S101, that is, the near-infrared detection data and EEG data of the subject can be collected while the display is used to provide the subject with the task block, thereby realizing data collection under the dual-modal paradigm.

[0082] In some other embodiments of the present application, the EEG data of the subject 50-400ms before the presentation of each digit in each trial is collected as baseline EEG data, and the EEG data 1100-2000ms after the presentation of each digit is collected as target EEG data, so that the changes in the EEG data of the subject when performing the tasks in each trial can be accurately analyzed based on this.

[0083] At the same time, the near-infrared detection data of the subject during the adjacent first time intervals is collected as the baseline near-infrared detection data, and the change in the near-infrared detection data during each task block relative to the baseline near-infrared detection data is collected as the target near-infrared detection data, so as to accurately analyze the changes in the blood oxygen response of the subject when performing the tasks of each trial.

[0084] In some other embodiments, the dual-modality data collected can be used for joint assessment of the subject's brain function.

[0085] Specifically, the processor can be used to obtain the subject's brain function evaluation results, especially the evaluation results of the subject's working memory state, based on the collected EEG data and near-infrared detection data of the subject.

[0086] Among them, the brain function assessment result is an objective biological indicator of the subject, which can assist the doctor in evaluating the mental state / brain function of the subject in combination with other subjective and objective indicators such as behavioral science. The brain function assessment result can be expressed in the form of a predicted value representing the normal degree of brain function, the level of concern that requires the doctor's special attention, etc.

[0087] In some other embodiments of the present application, a terminal for providing a dual-modality paradigm based on fNIRS and EEG is provided, such as Figure 3 The terminal 300 includes a processor 301 and a display 302, and the display 302 is used to display prompts about task blocks.

[0088] The terminal 300 is configured to use the processor 301 to sequentially execute a first preset number of task blocks for the subject at a preset first time interval, and sequentially execute a second preset number of trials in each task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period, presenting a fixation point in a variable second time period and receiving the subject's judgment result on whether the number of the current trial is the same as the number of the previous nth trial, n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900ms, so that each candidate time period maintains a predetermined proportion; wherein the near-infrared detection data of the subject during each task block and the adjacent first time interval is used for collaborative analysis with the EEG data in a preset third time period from before to after the presentation of each number in each trial to obtain the subject's brain function auxiliary assessment result.

[0089] For example, the terminal 300 may be various portable terminals such as a tablet computer, VR glasses, AR glasses, a mobile phone, or may be a terminal such as a desktop PC or a television.

[0090] In addition, the terminal 300 can also be configured to execute the collaborative interaction method for collecting near-infrared detection data and electroencephalogram data described in various embodiments of the present application.

[0091] In some other embodiments of the present application, a data acquisition system based on a dual-modal paradigm of fNIRS and EEG is provided, wherein the data acquisition system comprises at least an fNIRS device and an EEG device, and each device is used to perform data acquisition when executing the collaborative interaction method for acquiring near-infrared detection data and EEG data as described in the various embodiments of the present application.

[0092] The processor may be a processing device including one or more general-purpose processing devices, such as a microprocessor, a central processing unit (CPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0093] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).

[0094] The exemplary methods described in this application may be implemented at least in part by a machine or computer. In some embodiments, a computer-readable storage medium stores a computer program that, when executed, implements the collaborative interaction method for collecting near-infrared detection data and EEG data as described in various embodiments of this application.

[0095] In some other embodiments, a computer-readable storage medium is also provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, a processor provides a dual-modal paradigm of near-infrared detection data and electroencephalogram data, specifically including: executing a first preset number of task blocks in sequence for the subject at a preset first time interval, and executing a second preset number of trials in sequence in each task block, wherein each trial sequentially includes pseudo-randomly presenting a single number in a first time period, presenting a fixation point in a variable second time period, and receiving a judgment result from the subject as to whether the number of the current trial is the same as the number of the previous nth trial, where n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900 ms, so that each candidate time period maintains a predetermined proportion.

[0096] In addition, the computer program can also be configured to execute the collaborative interaction method for collecting near-infrared detection data and electroencephalogram data as described in the various embodiments of the present application.

[0097] The implementation of such a method may include software code, such as microcode, assembly language code, high-level language code, etc. Various software programming techniques may be used to create various programs or program modules. For example, a program portion or program module may be designed with or by means of Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules may be integrated into a computer system and / or a computer-readable medium. Such software code may include computer-readable instructions for executing various methods. The software code may form a part of a computer program product or a computer program module. In addition, in an example, the software code may be tangibly stored on one or more volatile, non-temporary or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (such as optical disks and digital video disks), cassettes, memory cards or memory sticks, random access memories (RAMs), read-only memories (ROMs), etc.

[0098] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0099] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the full scope of the equivalent forms of the attached claims and these claims.

[0100] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A collaborative interaction method for collecting near infrared detection data and electroencephalogram data, characterized in that: The collaborative interaction method comprises: Using a display, the subject sequentially performs a first preset number of task blocks at a preset first time interval, and sequentially performs a second preset number of trials in each task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period; receiving a judgment result of the subject on whether the number of the current trial is the same as the number of the previous n-th trial in the second time period, wherein n is 1 or 2, and each second time period in each task block is selected from a group of candidate time periods of more than 900 ms in a pseudo-random manner, so that each candidate time period maintains a predetermined ratio, and the sequence of single numbers appearing in the first preset number of task blocks is a preset pseudo-random sequence; The data collected during the collaborative interaction process includes the near-infrared detection data of the subject during each task block and the adjacent first time interval, and the EEG data in a preset third time period from before to after the presentation of each number in each trial.

2. The collaborative interaction method according to claim 1, characterized in that: A single number is pseudo-randomly presented during a first time period, specifically comprising: During a first time period, a target number or a non-target number is pseudo-randomly presented alternately as the single number, wherein the target number represents a number that is the same as the number in the nth previous trial, and the non-target number represents a number that is different from the number in the nth previous trial, wherein the ratio of the target number in each task block is 10%-50%.

3. The collaborative interaction method according to claim 1 or 2, characterized in that: Each candidate time period is selected from the range of 900-2100 ms, and the deviation between each candidate time period is 50-600 ms.

4. The collaborative interaction method according to claim 1 or 2, characterized in that: The first preset number is 1-9, the second preset number is 15-50, the group of candidate time periods longer than 900ms includes a first candidate time period, a second candidate time period and a third candidate time period, the first candidate time period is 957-1357ms, the second candidate time period is 1357-1657ms, and the third candidate time period is 1657-2157ms.

5. The collaborative interaction method according to claim 1 or 2, characterized in that: The same subject uses different pseudo-random sequences in different collaborative interaction processes, and the difficulty of each pseudo-random sequence is the same or similar.

6. The collaborative interaction method according to claim 1 or 2, characterized in that: It also includes, before executing the task block, executing the training block, specifically including: Performing a third preset number of trials for the subject using the display, each trial sequentially comprising pseudo-randomly presenting a single number during a first time period and presenting a fixation point during a second time period; receiving a determination result of whether the number in the current trial is the same as the second number before in the second time period by the examinee, wherein the third preset number is less than the second preset number; The value of n is determined according to the accuracy of the subject's judgment results in the training block, specifically including: when the accuracy of the subject's judgment results in the training block reaches a predetermined accuracy requirement, n is determined to be 2, otherwise n is 1.

7. The collaborative interaction method according to claim 6, characterized in that: The ratio of single numbers in the training block to the target number is less than the ratio of single numbers in each task block to the target number, the target number represents the same number as the previous n-th number, and the non-target number represents a number different from the previous n-th number.

8. The collaborative interaction method according to claim 1 or 2, characterized in that: The first preset number of task blocks includes a first task block and a second task block; The collaborative interaction method further includes: performing a second predetermined number of trials in sequence in each first task block, each trial sequentially comprising pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period; receiving a determination result of the subject in the second time period as to whether the number in the current trial is the same as the number in the first previous trial; performing a second predetermined number of trials in each second task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period and presenting a fixation point in a variable second time period; receiving a determination result from the subject in the second time period as to whether the number in the current trial is the same as the number in the second previous trial; The execution priority of the first task block is greater than the execution priority of the second task block.

9. The collaborative interaction method according to claim 1, characterized in that: The first time interval is 10-60s, and the first time period is 217-1017ms.

10. The collaborative interaction method according to claim 1 or 2, characterized in that: Each preset pseudo-random sequence includes a target sequence and an interference sequence, wherein the last digit of the target sequence is the same as the previous n-th digit, the first digit and the last digit of the interference sequence are the same as the target digit, and the number of digits included in the interference sequence is different from the number of digits included in the target sequence; The difficulty of each preset pseudo-random sequence is determined based on at least one of a proportion of a target sequence in the pseudo-random sequence, a proportion of an interference sequence, and a distribution of the target sequence in the pseudo-random sequence.

11. The collaborative interaction method according to claim 10, characterized in that: Each preset pseudo-random sequence also includes a continuous target sequence, wherein the continuous target sequence includes at least two adjacent target sequences; The difficulty of each preset pseudo-random sequence is determined based on at least one of the proportion of the target sequence in the pseudo-random sequence, the proportion of the interference sequence, the distribution of the target sequence in the pseudo-random sequence, and the proportion of the continuous target sequence.

12. The collaborative interaction method according to claim 11, characterized in that: The proportion of the target sequence in each preset pseudo-random sequence is 10%-50%, the proportion of the interference sequence in the pseudo-random sequence does not exceed the proportion of the target sequence, and the proportion of the continuous target sequence does not exceed half of the proportion of the interference sequence.

13. A terminal for providing a dual-modality paradigm based on fNIRS and EEG, characterized in that, The terminal is configured to utilize a processor to: The subject is instructed to perform a first preset number of task blocks in sequence at a preset first time interval, and a second preset number of trials are performed in sequence in each task block, wherein each trial sequentially includes pseudo-randomly presenting a single number in a first time period, presenting a fixation point in a variable second time period, and receiving a judgment result of the subject as to whether the number of the current trial is the same as the number of the previous n-th trial, where n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900 ms, so that each candidate time period maintains a predetermined ratio; Among them, the near-infrared detection data of the subject during each task block and the adjacent first time interval is used for collaborative analysis with the EEG data in a preset third time period from before to after the presentation of each number in each trial to obtain the auxiliary assessment result of the subject's brain function.

14. A data acquisition system based on a dual-modality paradigm of fNIRS and EEG, characterized in that: The data acquisition system includes at least an fNIRS device and an EEG device, each device being used to collect data when executing the collaborative interaction method for collecting near-infrared detection data and EEG data as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the collaborative interaction method for collecting near-infrared detection data and electroencephalogram data as described in any one of claims 1 to 12 is implemented.

16. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, and when the computer program is executed, the processor provides a near infrared detection data and an electroencephalogram data dual-modal paradigm, specifically including: The subject performs a first preset number of task blocks in sequence at a preset first time interval, and performs a second preset number of trials in sequence in each task block, each trial sequentially including pseudo-randomly presenting a single number in a first time period, presenting a fixation point in a variable second time period and receiving the subject's judgment result on whether the number of the current trial is the same as the number of the previous nth trial, where n is 1 or 2, and each second time period in each task block is pseudo-randomly selected from a group of candidate time periods of more than 900ms, so that each candidate time period maintains a predetermined proportion.

Images